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Framework for Festschrift Essays 


The overall aim of this volume of essays is to honor Frank B. 
Murray’s commitments to empirically-based quality assurance and to 
the development of increasingly effective systems of quality control in 
educator preparation programs. 

As the editors approached the authors with the invitation to 
contribute an essay, two characteristic aspects of Frank’s own thought 
were mentioned to provide overall direction to the collection. The 
first was Frank’s consistent ability to see around the corner into 
the future; the present is of interest not only for its own sake, but 
for its dynamic possibilities. We suggested something of a ‘future 
trends’ orientation. The second characteristic of Frank’s thinking 
that we noted was the knack for seeing the ‘big picture’ of issues and 
educational processes as they connected to small, immediate, concrete 
actions. We asked the authors to consider writing with a view toward 
identifying an important trend in quality assurance, quality control, 
or measurement, speculating on the potential consequences of the 
trend, and suggesting one or two concrete policy or programmatic 
actions that should be tried now. Each author was, of course, invited 
to follow her or his own muse. The overall idea, however, was that the 
set of essays, collectively, might suggest an agenda for the future of 
quality assurance and accreditation. We believe you will be informed 
and challenged by the provocative essays that follow. Each author 
has a distinguished record as a leader in higher education and quality 
assurance. The essays clearly are shaped by the particular perspective 
the author holds on quality assurance. 

Mark LaCelle-Peterson and Diana W. Rigden 

Editors 



Foreword 


What can I possibly say to capture as well as honor Frank 
Murray’s career in teacher education? How does one underscore 
Frank’s many accomplishments and his impact across the years, 
including his leadership of the Teacher Education Accreditation 
Council (TEAC) and the central role he has played in structuring 
the teacher education accreditation process of the future under the 
Council for the Accreditation of Educator Preparation (CAEP)? These 
are the questions I have been asking myself lately on many of my daily 
morning walks. 

As commonly understood, the purpose of a festschrift is to publicly 
honor a living person of undeniable achievement by presenting 
scholarly pieces related to the individual’s work. (I can hear Frank 
right now chuckling and saying “Thank goodness she added the living 
part!”) In the case of this particular festschrift, the included scholarly 
works are centered on the progression of accreditation, both in a 
general sense, and more specifically within teacher education, and the 
future questions and possible solutions that the field must address. 

Frank Murray’s career started with a dual undergraduate major 
in philosophy and mathematics (St. John’s University), then included 
being a junior high school math teacher followed by a return to the 
academy and to his focus on developmental psychology (Johns 
Hopkins University) and finally to academic positions including a 
lengthy tenure as Dean of the University of Delaware, College of 
Education. Throughout his academic career, Frank has provided the 
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leadership for many prominent initiatives (e.g., Homes Group, Holmes 
Partnership, Project 30, Project 30 Alliance) and organizations 
(Piaget Society, Divisions 15 and 7 of the American Psychological 
Association) as well as serving terms as an editor of scholarly journals 
( Genetic Epistemologist, American Educational Research Journal). 
Each aspect of his career exemplifies the dual contexts within which 
Frank operates: first, the visionary as a critical thinker and, second, 
the teacher educator existing within a practical world. Those of us 
in teacher education have been the beneficiaries of Frank’s career 
progression as he worked to alter our understanding of the purpose 
and outcomes of accreditation. For his efforts on our behalf, Frank has 
been recognized and honored and, as recently as 2009, he was named 
one of only 60 distinguished members of the Laureate Chapter, Kappa 
Delta Phi international education honor society. Yet with all of these 
accomplishments, I know that the honor we are bestowing with this 
festschrift will be one of great pride for Frank. 

The authors of the essays in this volume have chosen their own 
paths to honoring Frank’s work. For instance, Jon Wergin begins 
his essay by questioning how we go about recognizing quality and 
how we can publicly communicate the aspects of quality to others. 
While Jon is referring to quality within a programmatic framework, 
the composite works within this festschrift help to illustrate the 
essence of quality within the accreditation arena and speak directly 
to the leadership of Frank Murray. Numerous references are made to 
traits of Frank’s leadership style as “visionary/futuristic,” leading by 
example, and naturally inquisitive, but my favorites include Judith 
Eaton who presents Frank as “resolute in the face of challenges” and 
Don Warren who characterizes him as “deliberately rowing against 


strong currents.” Perhaps all of this is best summed up by Dan Fallon’s 
pronouncement that “Rational curiosity lies at the center of Frank 
Murray’s being.” 

Through more than a decade of watching Frank lead meetings, 
present the TEAC message, or work to change policy, I have found 
that Frank educates, informs, and inspires me each time he asks a 
question, reflects back what has been said, or expands the listener’s 
understanding of the issue at hand. Like the authors of this festschrift 
and those of you who have known Frank over the years, my interactions 
with Frank have increased my confidence that the teacher education 
profession will meet the challenges ahead. His role in the changing 
landscape of accreditation has forced us to confront ourselves (as 
well as our critics), ask the right questions, and find ways to publicly 
communicate the evidence we find to support our answers. I have 
no doubt that Frank would say that his accomplishments have been 
enhanced by the many interactions and collaborations he has had 
with competent professionals across the education spectrum. The 
talent, knowledge, commitment, and influence of each author in this 
festschrift are formidable. Each essay extends the reader’s thinking 
on the history, current state, or future of accreditation and each, in its 
own right, honors the life-work of our friend and colleague, Dr. Frank 
Murray. 

All so often when one person is central to a new initiative it all 
seems to fade away when that particular individual relinquishes the 
helm, but the TEAC house that Frank and the other co-founders built 
has been based on Frank’s honest belief that teacher education had 
a story to tell and that, when programs are given the opportunity 
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and the structure to tell their individual program stories through the 
accreditation process, the entire field of teacher education would shine. 
To Frank’s and the other founders’ credit the structure will persist and 
the accumulating stories will continue. There can be no better way to 
extend our admiration and appreciation of Frank Murray’s life-long 
endeavors then to honor him with the presentation of this festschrift. 

Sandra B. Cohen 



Knowing by Asking: 

Frank B. Murray’s Life of Inquiry 


— Daniel Fallon 


Frank and I belong to the generation that came of age in the 
1950’s. With horizons expanding during a happy post war prosperity, 
our parents expected us to go to college, even though they had only the 
vaguest notion of what that meant. Both Frank and I accepted the role 
laid out for us, but neither of us was prepared to navigate the confusing 
process of choosing an academic future. Frank’s perceptiveness 
allowed him to see the shallowness of myriad claims by competing 
admissions offices. He responded in a way that was to characterize the 
rest of his intellectual life. 

At one of those frenetic college fairs in a high school gymnasium 
somewhere in Brooklyn, Frank noticed a quiet table in a corner 
and sought it out. The recruitment tool there was a filmstrip, even 
by the standards of the day regarded by most observers as a rapidly 
retrogressing technology. The first few frames featured a frustrated 
student at the college headed towards the administration building. 
Upon encountering the Registrar, the student throws his books on the 
counter, saying, “I’m tired of reading these irrelevant old things. I’m 
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not learning anything useful. I want to withdraw and I want my money 
back.” The Registrar proceeds to accommodate the student’s request, 
finding his records and passing him some paperwork. As the student, 
now a bit calmer, focuses on filling out the forms, the Registrar 
nonchalantly asks, “You said the readings are useless?” “Yes” answers 
the student. A short pause precedes the Registrar’s next question. 
“How do you know that?” There follows an iconic Socratic dialog that 
culminates in the student picking up his books and heading back to 
class. 

Frank was hooked. He picked up an application and was enrolled 
the following fall at St. John’s College in Annapolis, Maryland. After 
graduation he taught Biology for three years to middle school students 
in the Baltimore Public School system. A Master of Arts in Teaching 
degree at Johns Hopkins University ensued before he was recruited 
into the Ph.D. program in developmental psychology at Hopkins. 

Frank and I first became acquainted in 1985 through efforts 
to secure a grant from Carnegie Corporation of New York. I was 
president of the national association of arts and sciences deans, the 
Council of Colleges of Arts and Sciences, and in planning the program 
for the annual meeting I included a session on the value of cooperation 
between colleges of arts and sciences and colleges of education in the 
service of strengthening teacher education. Sensing interest in this 
theme from Carnegie, I began to plan a grant proposal, but I needed 
to have a partner from education, so I approached the American 
Association for Colleges of Teacher Education, whose president 
recommended Frank Murray, then Dean of the College of Education 
at the University of Delaware. Frank and I conferred and ultimately 
agreed upon a framework containing five elements: (1) subject matter 


knowledge; (2) pedagogical content knowledge; (3) liberal and general 
education; (4) cultural perspectives, and (5) recruitment into teaching 
of underrepresented minorities. We got the grant and launched an 
effort we called Project 30, named for the 30 universities chosen 
through a competition to participate. 

Frank’s contribution to the Project 30 idea became the seminal 
feature of the initiative. He turned the national meetings into variants 
of the analytical reading and seminars that characterize how St. 
John’s College develops critical thinking. Provocative outcomes of 
intellectual inquiry, such as essays, stories, or treatises, were assigned 
well in advance. Seminar groups were then formed of no more than 12 
persons each, which then explored the central themes in the assigned 
reading, focusing on their relationship to the enterprise of teaching 
and the education of teachers. Frank’s advice to each seminar leader 
was to begin with the question: “What is this reading about?” Once 
these fountains of inquiry were flowing, it was impossible to tell which 
participants came from colleges of education and which from colleges 
of arts and sciences. Intellectual rigor poured from these seminars 
not only into the rest of the meeting, but onto the home campus of 
each participant in ways that resulted in significant reform at the 
participating institutions. 

Rational curiosity lies at the center of Frank Murray’s being. For 
Frank, learning is knowing by asking, and teaching is asking about 
knowing. It is his relentless inquisitiveness that has propelled his 
scholarship, enlightened those around him and led him into public 
service via accreditation. In 1997 Frank approached me about his 
interest in accepting an invitation extended through the Council 
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of Independent Colleges (CIC) to design an accreditation system. 
Addressing the challenge for teacher education Frank wanted to ask 
the question, “Can the program prove through persuasive evidence 
that its graduates actually possess the attributes and skills that the 
program claims it will engender in its beginning students?” Of 
course, Frank had zeroed in on the heart of the matter. This was not a 
question that concerned the then prevailing national system of teacher 
education accreditation, which like most accreditation systems of the 
time measured program inputs such as characteristics of entering 
students, size of library, claims for courses in the form of syllabi, etc. 
For Frank, more important was a question this approach seemingly 
ignored, namely, are the graduates successful teachers? 

Frank asked me to read an essay on accreditation that had been 
written for a project at Columbia University (Graham, Lyman & Trow, 
1995)- The central thesis was that accreditation for postsecondary 
education was dysfunctional because it attempted to combine 
two contradictory purposes: external accountability and internal 
accountability. The former is driven by incentives to assure the public 
that the institution is doing well, and thus motivates it to stress 
its strengths and overlook its weaknesses. The latter is driven by 
incentives to help the institution get better and thus motivates it to 
gloss over its strengths while focusing on its weaknesses. 

The authors of the essay offered a solution to the dilemma of polar 
conflicting priorities in traditional accreditation. Institutions should 
turn external accountability over to agencies outside of the institution, 
such as trustees, accounting firms, and state and federal agencies, 
leaving the institution to focus exclusively on internal accountability. 


The mechanism proposed by the authors for securing strong internal 
accountability was a periodic audit of the institution’s quality control 
procedures. The purpose of the audit, the authors concluded should be 
“...to determine whether the institution has in place procedures and 
practices that enable it to learn about itself, its weaknesses as well as 
its strengths, and to use that knowledge to address those weaknesses.” 

A problem in applying the analytical wisdom of the Columbia 
University essay to teacher education was that the authors came 
down squarely opposed to specialized accreditation such as teacher 
education. They asserted that incentives in specialized accreditation 
led colleagues to recommend such amenities as higher salaries, bigger 
operating budgets, more faculty positions, and better facilities rather 
than to root out weaknesses. Furthermore, such accreditation bodies 
practically never recommended withdrawing accreditation from 
programs in large recognized institutions, instead inappropriately 
putting pressure on smaller vulnerable institutions to change their 
missions . The authors concluded with an unequivocal recommendation : 
“The unit of analysis for accreditation should be the institution itself, 
not some separately designated program, school, or department.” The 
authors acknowledged, however, that specialized accreditation was 
not likely to disappear anytime soon. This allowed Frank to focus on 
an inconspicuous sentence, “If specialized accreditation is to continue, 
it should shift its activities to strengthened internal reviews focused 
on learning.” This became the germ from which sprouted the Teacher 
Education Accreditation Council (TEAC). 

At his invitation, I joined Frank, along with a few others, for 
several sessions, each for at least a day, in which we explored how an 
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alternative “outcomes-based” accreditation system might work. Over 
time we settled on an approach that used as a model the guidelines for 
scholarly papers for evidence-based experimental journals, consisting 
of four parts: Introduction, Method, Results, and Discussion. This 
became the foundation upon which the process developed and refined 
by TEAC was based. 

Of course, Frank’s approach to accreditation of teacher education 
was driven primarily by his focus on assembling evidence and using 
reason to answer essential and central questions. It was an exemplary 
instance of knowing by asking. But his inevitable focus on outcomes has 
been happily reinforced by developments in the underlying political 
economy of the United States and other advanced economies. The 
United States crossed the threshold to mass higher education in the 
late 1960’s, and the consequences of larger amounts of human capital 
have been profound. Increasing the density of educated persons in a 
society produces unforeseen effects. It changes the rules of the game 
for everyone. As the economy began to produce wealth more from 
knowledge, information, and services than from the old mainstays, 
agriculture, manufacturing, and heavy industry, the nation’s spotlight 
began to focus on the production of human capital. Without increasing 
amounts of high quality human capital the new political economy will 
falter, leading to lower standards of living and decreased quality of 
life. 


When higher education was the preserve of a relatively small 
proportion of citizens, logical and normative arguments to support the 
cause of higher education were sufficient. Academically elite students 
were selected, successful degree seekers were granted access to a 
curriculum that stressed historic shared values, and employers and 


post graduate schools accepted the graduates with few reservations. 
But when the health of the political economy depends upon well- 
educated knowledge workers, the proportion of citizens attending 
post-secondary institutions is large, and the financial investment of 
society as a whole is at stake, then logical and normative arguments 
must be supplemented with empirical ones. The empirical argument 
that trumps all others is evidence of outcomes. 

The reauthorization in 2001 of the Elementary and Secondary 
Education Act, commonly called “No Child Left Behind,” is perhaps the 
most visible and significant policy initiative marking a shift towards 
outcomes-based measures of progress. Using empirical benchmarks 
based on standardized tests to measure “adequate yearly progress” of 
pupils in the schools was both a significant turning point and politically 
very popular. The bill passed in the House of Representatives by a 
vote of 384 to 45, and in the Senate by a vote of 91 to 8. Not much 
later in 2005 then Secretary of Education Margaret Spellings formed 
a Commission on the Future of Higher Education, which called on 
postsecondary institutions to find ways of empirically verifying that 
their graduates actually know what the institutions assert to have 
taught them. The rise of instruments like the Collegiate Learning 
Assessment that claim to measure the value added to students by the 
liberal education provided by their higher education also reflects this 
societal demand. 

Frank Murray’s leadership had already by the beginning of the 
21st Century put TEAC at the forefront of the parade toward empirical 
verification of outcomes. TEAC was providing persuasive arguments 
for the validity of programs in teacher education. Not only the typically 
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small institutions represented by the CIC but also in significant 
numbers well-established prestigious research universities chose 
to seek accreditation of their teacher education programs through 
TEAC. Just as intended, the audit process pioneered by TEAC helped 
faculty, students, and administrators identify weaknesses they had 
been overlooking, clarified the central mission of the program, and 
led those responsible for the program to discover means to strengthen 
it. The positive program-focused success of TEAC accreditation led 
to enthusiastic endorsement of the process by teacher educators and 
gradually also by policy makers at the local, state, and federal level. 
Evidence of TEAC’s success included not only approval by the federal 
government and the national organization overseeing accreditation, 
but also by entreaties from the older national accreditor for teacher 
education that ultimately led to a merger with TEAC to form the 
Council for the Accreditation of Educator Preparation. 

Frank has pointed to the future of accreditation of teacher 
education by asking himself, “What have we learned through the 
TEAC experience?” (Murray, 2010). His answer is that assembling 
empirical evidence through audits opens up a variety of important 
research questions. Some of those can be partially answered by the 
evidence itself. Examples he cites include the fact that programs in 
teacher education are not “cash cows” for their sponsoring colleges and 
universities as is commonly and very incorrectly alleged, and, more 
intriguing, the hint that empirically supported success in teaching, 
what we might call successful clinical practice, is correlated at close 
to zero with any measure of academic achievement. Other recent 
research on successful and unsuccessful teachers appears consistent 
with this observation (e.g., Boyd, Grossman, Lankford, Loeb, and 
Wyckoff, 2009). The hints produced by accreditation evidence, 


however, ultimately must be secured through carefully conducted 
empirical research. 

The future of accreditation, particularly of the kind successfully 
developed by TEAC, is surely in fostering empirical research. The 
prevailing “research” literature on teacher education is deep and varied 
in normative propositions, reasonably solid in logical reasoning, and 
virtually bereft of foundational scientific empirical analysis. When 
Andrew Wayne and I reviewed more than 500 studies on teacher 
education for the American Council on Education in 1998 we found 
fewer than 20 that we felt were empirically informative (American 
Council on Education, 1999). Similarly, Wilson, Floden and Ferrini- 
Mundy (2001) reviewed hundreds of research articles on teacher 
education and found only 50 that they determined conveyed useful 
empirical information. The enterprise of teacher education urgently 
needs good scientific empirical research to clarify what elements in 
a teacher education program significantly contribute to preparing 
an effective teacher. Frank has given us an excellent example of 
how accreditation can give birth to a research program. Consider 
his observation of the seeming lack of correlation between strong 
academic achievement and successful clinical practice. 

What TEAC accreditation has provided is a set of observations 
that lead to an intriguing hypothesis. Let’s assume, as we think about 
research, that it is true that measures of academic achievement are 
correlated at zero with measures of clinical practice. We begin with 
the fact that a zero correlation is not a negative correlation. Therefore, 
we could imagine a 2 x 2 contingency table in which the two columns 
were high and low academic achievement and the two rows were high 
and low clinical practice scores. A zero correlation would posit that 
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a random selection of novice teachers would place a roughly equal 
number of teachers in each quadrant. Which of these quadrants or 
columns or rows would we want to maximize as teacher educators? 
Would it be sufficient merely to maximize the entire row of successful 
clinical practice? If so, what would it mean for a pupil to have a highly 
successful teacher who was not academically accomplished? Would 
the teacher successfully teach the pupil wrong facts, e.g., that Theodore 
Roosevelt rescued the nation from the great depression? 

Similar kinds of questions might be directed at each of the 
quadrants. One might assume that ultimately as teacher educators 
we would want to maximize just one quadrant: high academic 
achievement and high clinical practice. But empirical research 
often confounds us with facts that require further investigation. For 
example, in a persuasive empirical study by Monk (1994) the number 
of college level mathematics courses taken by the teacher served as 
a measure of mathematical knowledge, and scores on the National 
Assessment of Educational Progress provided a measure of student 
learning. Student learning of mathematics increased monotonically 
with the number of courses taken by the teacher, but only up to an 
asymptote of about five courses. A subsequent empirical study by 
Rowan, Correnti, and Miller (2002) indicated that students taught 
by teachers who held an advanced degree in mathematics learned 
less than those taught by teachers who had taken fewer courses in 
mathematics. These are important findings for understanding the 
nature of teacher learning as it relates to student learning, but they 
could not have been persuasively revealed by logical argument alone. 
Empirical study was essential. Furthermore, these empirical findings 
call into question a generic normative and logical assertion that 


teachers of subject matter will improve their teaching by completing 
advanced academic work in the field of study they will be teaching. 
The findings do not necessarily contradict this assertion, but suggest 
that important qualifications may need to be added. Before we can 
proceed to determine how to qualify the assertion, however, further 
empirical investigation is required. 

In pursuing a life of inquiry through knowing by asking, 
Frank Murray has pointed us toward the importance of empirical 
knowledge. He has used that knowledge to foster a mode of teacher 
education accreditation based upon verifiable evidence. By bringing 
this mode of inquiry to the fore, he has pointed toward a future in 
which teacher education accreditation becomes the fountainhead for a 
new era of urgently required empirical research. Perhaps his academic 
training in psychology was congenial in strengthening his convictions 
about evidence. In a classic 1890 text, Principles of Psychology, the 
American philosopher William James mused about whether he could 
adopt different frames of reference when thinking about how he 
might be a person in the world. Each of these frames, James (1890, 
pp. 309 310) wrote, constituted a kind of “self,” providing the values 
and coherence appropriate to a complete personality. He wondered 
whether, for example, “I could be ... a great athlete and make a million 
a year, be a wit, a bon vivant and a lady killer, as well as a philosopher, 
a philanthropist ... and saint.” 

His analysis led James to conclude that he could, in fact, only be 
who he actually was. If he had different “selves,” he reasoned, they 
could only be empirically defined, that is, by the words and deeds that 
he himself professed. And so he arrived at the conclusion that “I am 
often confronted by the necessity of standing by one of my empirical 
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selves and relinquishing the rest.. .all other selves thereupon become 
unreal, but the fortunes of this self are real. Its failures are real failures, 
its triumphs real triumphs....” For Frank Murray, an empirical self is 
his real self, and it has enriched us all with a multitude of pregnant 
questions. 
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Time, Our Times, and TEAC 


— Donald Warren 


Of the relevant disciplines, history, possibly only history, can 
adequately explain Frank Murray’s significance within the Teacher 
Education Accreditation Council and research on intentional teacher 
preparation generally. It illuminates TEAC advances that otherwise 
could be missed, a composite policy and practice trajectory launched by 
ideas Frank fashioned, tested, and refined across a decade. Expanding 
the point, one might argue that history offers the most trustworthy 
avenue to understanding program quality, despite obvious limitations. 
Like any framework for investigating program performance, the 
historical perspective is context-driven, unstable, and fails validity 
and reliability tests with the passage of time. Certainly, history is 
the approach most deeply embedded in TEAC principles. All their 
leading questions about programs ( What have teacher candidates 
learned? How do faculty know their program is effective? What 
have they and their institutions learned from their successes and 
failures? What evidence suggests they have amended their program 
accordingly?) evoke varieties of past evidence. In some cases faculty 
claim a need to visit anew their program’s origins and those of the 
college or university itself. Reconstructions of institutional and 
program development are not merely fillers in a self-study document, 
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and are definitely much more than exercises in nostalgia. They are 
searches for documentation of the inevitable serial battles over change 
and purpose. That self-professed track record of cultural devolution 
and achievement lends confidence in a program. The story is always 
mixed, never simply linear or progressive. Whatever its direction, the 
history provides empirical encouragement as faculty and institutions 
struggle with the ravages of time and circumstance: fluctuating 
budgets; aging structures and technologies; faculty and staff turnover; 
shifting student identities; altering local, state, and federal policies; 
and varying social and cultural dynamics, arenas in and through which 
schools and teachers operate and, for good or ill, leave their marks. 
The inevitability of change sharpens the urgency of TEAC’s purposes, 
unsettles program confidence, even if belatedly, and impels historical 
orientations, assuming the aim is teacher education and not training 
alone. TEAC intends to promote programs embedded in pervasive, 
untimid curiosity, a culture of inquiry satisfied by evidence, however 
unsettling initially. Toward this imperative, Frank Murray repeatedly 
has grasped high expectations, pushing others so inclined to reach 
farther still. Understood as reconstructed momentum, a running start 
on the present, history delivers a useful way to honor his achievements 
and TEAC’s. 

He, and TEAC itself, often and deliberately rowed against 
strong currents. Teacher preparation traditions in the United States 
and elsewhere have long revealed a weakness for fads, status envy, 
and depleted metaphors, all at the expense of commitments to 
independent foresight and grounded outcomes. The longing for 
homogeneity, visible in the one-best-system mentality dominating 
school and teacher preparation reform, can be traced to demographic 
origins, the Jeffersonian ideal of an agricultural society (Lauzon 2011). 


Bureaucracy, the Progressive Era’s organizational reform platform, 
meshed smoothly with the rural ethos and thus propelled uniformity 
as desirable policy (Labaree 1997). 

Institutionalized, the inclinations infected other professional 
fields as well. For one, look to U.S. law schools. The crisis in American 
legal preparation recently announced by editors of The New York 
Times (2011, 26 November) accuses law schools of being mired “in 
an outdated instructional and business model” (p. A16). As Daniel 
T. Rodgers argues in Age of Fracture (2011), this stubborn inertia 
stems from an addiction to either-or thinking. It mirrors precisely 
our age of fracture formed by social and intellectual atomization, and 
the dangerous failure to appreciate the educative magic of purposes 
and words. We need law schools, The Times editorialized, that can 
organize themselves, deliver instruction, and in short think in the 
future tense “to align what and how they teach to what legal practice 
now entails and what individuals and institutions need - like many 
more lawyers who can serve as advocates of the poor and middle 
class” (p. A16). By implication, advocacy roles of any sort lack political 
capital in an age of fracture, a contemporary mood inhibitor of reform 
options the editors overlook. Nevertheless, they soldier on. In addition 
to advocates and counselors, they propose, American society needs 
lawyers well prepared for multiple other future roles as “negotiators 
and deal-shapers, and problem solvers.” To these ends, according to 
The Times, we need law schools rigorously committed to both-and 
modes of inclusive and open-ended planning, teaching, research, and 
organization. Key to success will be effective teaching, theory and 
practice wed, with all the intimacy the verb implies, and ideas and 
skills tightly laced. 
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Such goals remain problematic when confrontation, competition, 
and markets delimit language and possibility. Our reigning ideals, 
constituting ways of thinking, nothing more or less, seek winners, 
losers, and other fragments, rather than the unifying diversities of 
teaching and learning. Market metaphors, Rodgers (2011) observes, 
now lie unquestioned yet explicit in scholarly discourse, political 
rhetoric, media punditry, and even happy hour chats. Wall Street 
enterprises and universities alike seek “branding” advantages. He 
blames the intellectual retreat of economics, the dismal science, for 
the ironic development. Market metaphors snap cultural glue, raising 
individualization as a facade covering a contemporary manifestation 
of group thinking, our version of the herd instinct. We do not know yet 
how to back out of this cul-de-sac, Rodgers warns, but to synchronize 
creativity and intellectual rigor and advance them in the education of 
the public, we urgently need to try. 

Do similarly alienating modes of thought constrain teacher 
education and imply a research agenda of new business? If so, 
accreditors should be asking whether teacher preparation ought 
to be more out of joint with its times, not scampering to echo the 
language of exclusion. The repetitious character of its reform history 
suggests deep-seated habits: a rocky career of recycled initiatives and 
persistent social conformity, hints of systemic pathological obedience. 
Unreliable funding levels and governance structures arranged 
primarily by accretion help explain both. From the outset, teacher 
work and preparation were linked through policy and practice, even 
when they hardly met. The story began over three centuries ago at 
colonial colleges like Harvard (arguably the nation’s first teacher 
preparation institution) and the academies whose histories Ted 
Sizer (1964) traced. Graduates or young men pausing in their studies 


to contemplate what they wanted to do with their lives sometimes 
agreed to become schoolmasters. Typically, they did not last long, the 
job proving ill-suited to reflection. Such was the painful discovery of 
John Adams (Harvard) and Henry Barnard (Yale), two well-known 
temporary teachers. Both famously landed in politics, the former by 
design and the latter as the accidental first United States Commissioner 
of Education (MacMullen 1991). One of Adams’ charges recalled he 
seemed “absorbed” and given to much frowning (McCullough 2001, 
p. 38). Worth noting, the colleges and academies offered no programs 
for future teachers. Rather, a common assumption intervened, namely 
that anyone with an academic foundation, however incomplete, could 
handle a classroom and the prevailing school curriculum. 

As schools evolved in the nineteenth century, so did their isolation 
by institutional type, erecting conceptual and policy silos straining 
developmental links among common schools, high schools, colleges, 
and later universities. The introduction of kindergartens and junior 
high schools as separate (but not equal) professional and research 
fields blurred organizational lines but left the bureaucratic maze 
intact (Garrison 2006). Categorizations of teachers by assignment, 
salary, and status followed suit. High school teachers migrated from 
colleges and academies, which still lacked preparation programs; 
common school teachers tended to bring that level of preparation to 
their classrooms. In both cases, local school committees, citizens, and 
parents exerted control, taking stock of whether a teacher delivered 
the goods. Two general criteria tended to be applied, student behavior 
and rote learning, the latter proven by written tests, recitations, 
and classroom observations conducted by committees, parents, and 
occasional observers. Spelling bees and public oral quizzes displayed 
young scholars’ achievements. These intensely local arrangements left 
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a potential dysfunction unaddressed. Could they satisfy the republic’s 
need for literate, informed citizens? Yes, they could, but only within 
narrow variations among schooling opportunities locally. The 
concern inspired antebellum efforts to establish county supervisory 
structures and state agencies to broaden the scope of local control, 
but these developments occurred primarily in northern and western 
states, North Carolina being a notable exception. In the 1860s, the 
trend moved to the federal level. The Civil War, Louis Menand (2001) 
observes, permitted policy debates, as over against sentimental 
ruminations, on a national system of schooling, proffering a new 
political entity, government by a federal partnership to collate local, 
state, and national responsibilities, not by competing levels of power. 

Similar constraints shaped the beginnings of formal teacher 
preparation. Early on, teacher seminaries emerged to offer training 
for candidates headed to common schools. In the late nineteenth 
and early twentieth centuries, high schools in large to midsize cities 
expanded their still forming curricula to include teacher preparation. 
These several institutional types morphed into private, usually 
sectarian, and state institutions and then into normal or pedagogical 
colleges. Throughout this period, one should remember, the majority 
of elementary teachers brought little to no preservice preparation 
to their first jobs. With contracts secured, they acquired inservice 
training via summer institutes and periodic enrollment at a nearby 
institution. Money was a problem, or more precisely foregone 
earnings. Teacher salaries were too low to foster financial incentives 
for preservice matriculation. Not until the 1950s did the majority of 
American teachers hold bachelor degrees (Rury 1989; Sedlak 1989). 


This abbreviated sketch suggests linear growth, following the 


predictable arc of modernization. Teacher preparation programs 
became longer and more detailed; normal schools became degree 
granting four-year colleges and later regional universities. Their 
status, effectiveness, and efficacy, however, seemed locked in first 
gear. In a detailed comprehensive survey, Teachers College’s Edward 
S. Evenden (1933) found no reliable research that helped cull failed 
teacher preparation programs from successful models. A key reason, 
he surmised, was their uniformity. In actuality there was only one 
model; all were essentially alike, with curricula trending toward 
practical training. Over the next six or so decades, studies attempting 
to crown particular programs or even just those with national 
accreditation failed credibility tests, outcomes traceable to designs 
contaminated by halo effects. Missing after all these years, a culture 
of inquiry had yet to take root to nurture teacher preparation with 
educative potential. Programs, and the faculties who offered them, 
still struggled for respect and funding within their home institutions, 
from state legislatures and agencies, and from private donors. Here 
too modernization in the cause of national standards and “excellence” 
reinforced trends toward uniformity. For almost two centuries, from 
the time they first took notice, a recurring theme in news media has 
reported a story of ineptitude and low expectations (Labaree 1997). 
In recent years, broadsides from the U.S. Department of Education 
have joined the chorus of negative claims. This was the situation 
TEAC and Frank Murray entered, an intellectual and professional void 
nonetheless packed to the brim with other phenomena. 

Whether as heroes or scapegoats, fictionalized portrayals of 
teachers confirm disparaging popular and scholarly views of actual 
teachers. They appear as selfless, philanthropic, passive creatures, 
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lacking human nuance and spirit. Perhaps the job attracts docile 
personalities which no amount of training can unbend. More likely, 
some scholars argue, teaching became women’s work and bore the 
stain of women’s historically low social status (Clifford 1989; Rury 
1989). They could also be paid lower salaries than men could, a 
significant factor in calculations. The problem with this interpretation 
is that it ignores the liberation nineteenth-century women experienced 
in becoming teachers (Clifford 1989). A similar point can be made 
regarding the educational effects of teaching for African Americans 
and American Indians over the past century (Lawrence 2011). In these 
three instances teaching and teacher preparation have functioned 
as triggers of educative cultural dynamics. Some values, it turns 
out, elude quantification and even perceptive literary imaginations. 
That possibility points to another explanation of the longstanding 
derogatory treatment of teachers and teacher preparation. Perhaps it 
stems from a blended problem of misperception and mismeasurement, 
both errors attributable to a discouraging characteristic: As published, 
the history of teachers and teacher preparation lacks interest (Ogren 
2005). A dull, repetitious, and culturally insignificant story forms. 
Following in the footsteps of John Adams and Henry Barnard, most 
teachers come and go. They exit after three to five years, leaving little 
discernible imprint on students or schools (Labaree 2004; Evenden 
1933)- The careers of African American and American Indian teachers, 
as examples, require longer, more nuanced narratives. 

Cautioning against studies with too much data and not enough 
insight, historical and sociological analyses tend to agree that 
intentional teacher preparation, like the reigning perception of 
the profession it feeds, may have changed quantitatively but not 
structurally or essentially (Labaree 2004; Ogren 2005). From the 


early nineteenth century onwards, it has emanated from a stark 
dichotomy between subject matter and teaching methods. Twentieth 
century attempts to amass a research base for teacher preparation 
have only underscored the unnatural division between content and 
pedagogy, as though biology, or history, can be learned without 
simultaneously learning what knowledge is of most worth, how 
knowledge is acquired, and how it can be passed on to students. Willy- 
nilly, deliberately or not, mathematics departments deliver lessons 
in instructional methods. The reverse is true of the much maligned 
methods and student teacher courses that anchor teacher preparation 
programs. Math methods inevitably also teach mathematics and the 
discipline’s value. Divorced from content, classroom management 
becomes lessons in how to police students, an abstract and temporary 
arsenal at best. In the world of schools and classrooms, content and 
pedagogy are interwoven realities, a tapestry of sorts. The resolution 
of the conceptual and curriculum planning dilemma is not to reduce or 
eliminate one or the other focus but to insist on maintaining both and 
doing so with imagination, verve, and rigor. The aim is to empower 
teachers to empower their students (Labaree 1997). 

To this end, we want teacher preparation to be teacher education. 
Courses in reading instruction offer strategic skills, current research 
findings, cultural variations of literacy, and assessment tools, bundled 
to advance reading ability and love of literature. Likewise, educational 
psychology and human development introduce teacher candidates 
to the complexities of learning, disaggregated by life stages. History 
of education, arguably the oldest component of teacher preparation, 
enables prospective teachers to critique the persistent themes and 
problems of their profession and to tease out explanations for the 
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repetitious character of its past (Evenden 1933). History arms them 
against false dichotomies and inclinations to lapse into simplistic 
binary modes of thinking. The history of the history of education field 
suggests its practitioners have not been effective in meeting such 
expectations, and in any case, curriculum specialists jealously have 
opposed the goals as impractical (Evenden 1933; Ogren 2005). 

Typically ignored in these internecine battles and omitted from 
historical and sociological analyses is curiosity about links between 
education and culture, the dynamic structural, moral, and intellectual 
cohesions that bind people to each other as a community. Rather, 
pedagogical and discipline-based approaches have conflated education 
and schooling, leaving the former imprisoned within the latter and 
schooling disconnected from the forces of alienation and aspiration 
that propel agency and inspire struggles to endow work, leisure, 
politics, and common sense with meaning. Teacher education requires 
programs that equip candidates to understand their own times and 
those of their students, to be rooted contextually and existentially. 
Such conceptual complexities and professional imperatives limned 
the scene TEAC entered in the 1990s. A career-long practitioner of 
research on teaching and learning, with a take-no-prisoners preference 
for statistical analyses, Frank Murray thoroughly understood them. 

TEAC’s origins arose from a determination by the founders to 
complicate the discouraging narrative of teacher preparation with 
evidence and insight. They wanted an accreditation system that 
acknowledged the healthy variety of colleges and universities offering 
preparation programs. Whether directed toward large or small, public 
or independent, liberal arts colleges and multi-purpose universities, 
the essential questions to answer have to do with a program’s 


effectiveness and a faculty’s ability to gather, array, and analyze 
relevant data. Candidates need to be knowledgeable and confident; 
faculty and administrators need to draw their expectations of students 
from systematic, evidence-based research and apply valid and reliable 
measures of outcomes, as short-lived as they are. Under Frank 
Murray’s leadership, TEAC has been more successful in this effort 
than the founders had originally anticipated. One reason has been that 
TEAC extended the principles of continuous assured improvement 
in teacher preparation to its own system. The aim has been to learn 
non-defensively from experience, driven by commitments to test, 
qualitatively and quantitatively, and to experiment. 

Important work lies ahead. TEAC has not yet populated its board 
of directors, committees, and staff with people of color and cultural 
minorities. Historically black colleges and universities and American 
Indian institutions are notably absent from its list of accredited 
programs. This isolation must change. It confers on us an aura of 
narrow-mindedness and cowardice, an intentional irrelevance with 
respect to both current events in the world and the minds of teacher 
candidates and their students-to-be, who will contribute to the shaping 
of these events. Perhaps, TEAC can learn how to proceed through its 
planned ventures into Caribbean and Middle Eastern institutions. 
Here we will encounter not only language diversity but also unique 
traditions of indigenous teaching and learning. Necessarily, our 
audits will be contextualized historically. For one, they must be alert 
to residual effects of twentieth-century U. S. colonialism on schools 
and teacher preparation programs (Go 2008; Lawrence 2011). More 
pointedly than in the past, we will need to adapt specific culturally 
informed orientations in our assessments. As TEAC learns these 
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essential lessons, it can reach out legitimately and confidently to 
a wider range of diverse programs, seeking as it has in the past to 
expand the community of inquiring faculty and teachers. This unifying 
commitment is our counter to an age of fracture and a TEAC legacy 
worthy of Frank Murray’s pioneering achievements. 
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Five Essential Tensions in Accreditation 


— Jon F. Wergin 


For most of my professional life I’ve been fascinated with the 
term “quality.” What is it? How do you know it when you see it? How 
can you communicate what you know to others? Robert Persig, in 
his classic book Zen and the Art of Motorcycle Maintenance (1974), 
wrote, “Quality... You know what it is, yet you don’t know what it is” 
(p. 225). Persig had it exactly right. 

Higher education has had to deal with the quality conundrum 
only recently. Prior to the 1980’s educational quality was mostly about 
resources and reputation, and this is what accreditors mostly focused 
on. The more you had of each, the better you were. But then came 
what Bob Zemsky (1993) called the “loss of sanctuary”: with tuition 
steeply rising, and a college degree not the ticket to a better life that 
it once was, higher education was no longer immune from having to 
demonstrate public accountability and assure its usefulness to society. 

As the readers of this volume know, accreditation is an institution 
that is indigenous to the USA. Unlike virtually every other country, 
public accountability for quality is not a function of a government 
agency but of peer review. We are accountable to the larger society 
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through each other. But the loss of sanctuary that Zemsky referred 
to twenty years ago is being felt with increasing force in accreditation 
practice. Here’s why. Higher education in the United States 
demonstrates public accountability in three ways: governmental 
regulation, the marketplace, and peer review. 

Governmental regulation includes federal and state government 
and state coordinating and governing boards. Traditionally regulators 
ensure that higher education institutions are fiscally and socially 
responsible, that they meet appropriate safety and health standards, 
and that they offer educational programs that aren’t unnecessarily 
duplicative. When an institution attends to regulation, the goal is 
compliance. 

The marketplace, especially with the explosion of online learning, 
has increased the competition for students among educational 
producers. Institutions and programs that fail to adjust to a changing 
market put their own health and survival at risk. When an institution 
attends to the marketplace, the goal is competitive advantage. 

Of the three forms of public accountability, peer review is the only 
one that focuses on the quality and integrity of student learning itself, 
and it is the only one over which the institution and its faculty have 
any direct control. The collective faculty have traditionally been the 
ones responsible for maintaining program quality, and no one wants 
to leave that function to the government or to the marketplace. 

But increasingly, that is exactly what is happening. Government 
regulation, in particular from the U.S. Department of Education, 


has become far more intrusive in the past decade. Part of this 
is due to the abuses of the marketplace, where in the search for 
competitive advantage, for-profit universities have catered to the 
demands of student “customers” at the expense of quality control. As 
bureaucracies are wont to do, everyone then pays for the excesses of 
a few. But we can’t just blame government activism on the for-profit 
universities out there. We need to look inward as well. The fact is that 
peer review, the form of public accountability in which the institution 
should have the greatest vested interest, is also the weakest. We have 
struggled to develop ways of answering the questions I posed at the 
beginning of this chapter: What do we mean by academic quality? 
How do we know it when we see it? And most importantly, how do 
we communicate what we know to our publics? Such methods of peer 
review as accreditation and program review policies are widely seen 
by faculty as little more than rituals - bureaucratic hurdles to clear, 
necessary evils to put up with so that the real work of the academy 
can get done. In a study I led for the Pew Charitable Trusts about a 
dozen years ago (Wergin and Swingen, 2000), my colleagues and I 
surveyed the assessment practices of 130 institutions around the 
country, large and small, public and private. What we found, in a 
nutshell, was a widespread “compliance mentality,” as I called it: Find 
out what the accreditors want, give it to them, then go back to work. 
A lot of assessment was going on, but not much institutional learning. 
It was all quite discouraging. It was the rare institution that was able 
to use mandated assessment of any type in the service of improved 
professional practice and better student learning. 

In the midst of working through these depressing results, I met 
Frank Murray and TEAC and found hope. If this seems maudlin, 
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written only because this chapter is part of a festshrift, it nevertheless 
happens to be true. I saw captured in TEAC principles and policies the 
embodiment of what the few exemplary institutions in our sample were 
doing right. I wrote the following in an article for Change Magazine 
about the increasing importance of accreditation: 

I believe that alternative models such as TEAC, with [its] dual 
focus on collective responsibility and public transparency, 
[has] great promise. Not only does [it] result in greater internal 
ownership of quality assurance, [it] also responds positively to 
Congressional mandates for more openness about the process 
itself. (2005, p. 40) 

TEAC isn’t the answer to quality assurance, of course. No 
accreditation model is. But TEAC is able to negotiate several essential 
tensions of quality assurance better than most if not all of the 
alternative approaches. There are five essential tensions, I think: 

1. Outside-in - inside-out perspective. Making 

academic programs accountable to their constituencies has 
largely been an outside-in phenomenon. The importance 
of getting a largely insular academy to pay attention to 
constituent interests is obvious. But a shared commitment to 
accountability is also needed, a commitment in which both 
the members of the academy and its constituent communities 
have a stake. In the Pew study I alluded to earlier, the single 
most important factor discriminating the few institutions 
who used assessment successfully from the many that didn’t 
was whether or not the institution had evaluation policies 


that were flexible and decentralized (Wergin and Swingen, 
2000) - where units were invited to define for themselves 
the critical evaluation questions, the key stakeholders and 
sources of evidence, and the most appropriate analysis 
and interpretation procedures. Without inside-out energy, 
disappointment with compliance is almost inevitable. 

That disappointment in turn generates stricter and stricter 
accountability measures, creating an even greater imbalance 
between outside-in and inside-out. The TEAC model, with 
its emphasis on local definitions of quality and a public 
commitment to quality improvement within a framework 
of broad professional standards, keeps the tension more 
centered. 

2. Standardization - uniqueness. Balancing outside- 
in with inside-out requires attending to a second tension: 
finding a balance between definitions of quality into which 
all programs must fit, and those which recognize variations 
according to contextual variables, such as program and 
institutional mission and needs of the surrounding 
community. Graduates of a program that is jockeying for 
position in the national ratings may not be the best equipped 
to teach in under-resourced school districts. Programs 
contribute to the public good in different ways. The message 
of “we know what a quality program is and we’ll see if 
you measure up” must be balanced by “we think that you 
know what quality is, and we’ll see if you live up to your 
own standards.” TEAC does this by requiring candidate 
programs to write Inquiry Briefs in which they make certain 
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claims about what their graduates’ knowledge, skills, and 
dispositions should be, and then provide evidence to back up 
these claims. But not everything goes: program claims must 
adhere to certain core principles promulgated by TEAC and, 
in many cases, state professional standards of practice. 

Reductionist - essentialist. This is a slightly different 
way of capturing the tension between quantitative and 
qualitative methods in educational research. Some would 
argue that nothing is inherently ineffable about educational 
quality, that any such argument that quality simply can’t 
be defined is defensive and lazy and masks a reluctance to 
put one’s program on the line. If a quality exists, they would 
contend, then it can be empirically observed; if it can’t, 
then the quality is hopelessly abstract and therefore useless 
for assessment. Furthermore, if something is empirically 
observable then one ought to be able to attach a number to 
it and make comparative analysis possible. Others insist that 
a program should be evaluated for its essential qualities that 
don’t lend themselves easily to quantification. Using Eisner’s 
(1991) notion of “educational connoisseurship,” they argue 
that reducing complex realities to statistics takes the life 
out of a program and fails to convey the spirit of what it is 
doing. While not rejecting the utility of numbers altogether, 
essentialists find the most revealing source of information 
about a program to be the story it tells about itself. TEAC has 
struggled with this tension with only moderate success, in 
my opinion. In my experience, more often than not writers 
of Inquiry Briefs have succumbed to a compliance mentality, 


filling their Briefs with numbers they think TEAC wants to 
see, rather than building an authentic and compelling story 
about themselves, using evidence that is pulled from a variety 
of sources, using a variety of methods. 

4. Outcome-process focus. In the past twenty years both 
regional and specialized accreditors, virtually without 
exception, have gotten serious about requiring member 
institutions and programs to specify student learning 
outcomes, how they assess to these outcomes, and how they 
use assessment data to improve educational practice. Given 
the previous focus of accreditors on resources and reputation 
as markers of quality, this is a welcome shift. I’m troubled, 
however, about the extent to which a focus on outcomes has 
been reified. As with the reductionist/essentialist tension 
above, does anyone really know the complete set of important 
outcomes of a quality education, especially when preparing 
students for the complexities of professional practice? Every 
experienced practitioner is able to cite examples of exemplary 
professionals who excel in spite of - or maybe because of - 
the degree to which they flout conventional wisdom. Further, 
holding programs accountable for results discourages risk- 
taking. This is what happens when faculty members are held 
hostage to student ratings: they know that tinkering with the 
status quo, especially when it might create disorientation in 
the class, will likely lead to a drop in student satisfaction, at 
least temporarily. Faculty and programs need the freedom 
to experiment, and to fail. Accountability must focus not 
just on outcomes but on what programs learn from their 
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experiments, and what they do with that learning. TEAC could 
help achieve a better balance between process and outcome by 
focusing more squarely on organizational learning, and less 
on how well programs are able to justify their claims. 

5. Public accountability - program improvement. This is, I 
think, the most difficult tension to manage of all. Because 
the culture of accreditation is built upon peer review, it’s far 
easier and less painful to tell colleagues in other institutions 
what they need to do to improve than it is to tell them that, at 
the end of the day, they’re just not good enough. Accreditors 
have been pilloried relentlessly for their reluctance to deny 
accreditation even in the face of the most egregious violations 
of standards; and when accreditation is denied, the basis for 
the denial is rarely if ever related to poor educational quality, 
but to non-pedagogical concerns such as an unsustainable 
resource base. Most who serve on accreditation teams squirm 
at being examiners and gatekeepers; they are much more 
comfortable seeing themselves as colleagues offering friendly 
criticism when needed. TEAC, with its emphasis on the 
academic audit, makes it clear that the role of the audit team 
is to determine how program claims might be verified with 
available evidence, not to play “gotcha.” Fair enough; but who 
is responsible for making value judgments about the quality 
of the claims themselves? Does a program that sets high 
standards for itself, and then falls short, demonstrate lower 
quality than one that fully achieves more modest claims? I’ve 
seen this struggle firsthand as I’ve served on the TEAC panel: 
given the candidate program’s Inquiry Brief, and the audit 


of that Brief, the panel is charged with making a summative 
judgment of quality. Here is where all of the tensions I’ve 
described come into play. Does the relatively-stronger 
“inside-out” perspective that TEAC has make demonstration 
of public accountability more difficult? If program quality is 
at least partially context-dependent, what aspects of quality 
should not be? Similarly, if a program submits a qualitative, 
essentialist Inquiry Brief, what external standards apply? 
Doesn’t a program have to finally be held accountable for 
what its graduates can do? 

Five years ago, if someone had asked me how I’d respond to all 
of these questions, I would have had a ready answer: transparency. 
Serving the public good, I would have said, is having a process by 
which the program is an open book: it makes clear to its constituencies 
what its mission is and why this mission is important; it makes specific 
claims about what it aspires for its students, how these claims relate 
to program mission, and how it knows whether or not students have 
achieved these claims; and it shows what the program faculty do to 
improve their educational practice based on the evidence they collect. 
This is the TEAC process, as publicly transparent as they come. I 
would have gone further and asserted that quality lies in the eye of 
the beholder, that it’s up to prospective students and their significant 
others to decide if their investment in that program would be 
worthwhile, given the kind of teacher or administrator they want to 
be. Thus, a program’s (and by extension, TEAC’s) accountability lies in 
the degree to which it provides authentic information of the sort that 
allows informed judgments to be made. 
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While I don’t forswear what I would have said five years ago, I’ve 
come to the conclusion that TEAC’s and other accreditors’ responsibility 
to the public good ought to go further than that. “Informed judgments” 
aren’t always that informed - at least in the way we professionals think 
they should be. The top criteria that most prospective students use 
in the uncertain economic climate of today, namely convenience and 
cost, are highly pragmatic. Accreditation status is one of the criteria, 
to be sure, but only if accreditation is required for licensure in their 
state. Besides, if a program is TEAC-accredited, they ask, doesn’t that 
mean that it’s good? 

Well, maybe, maybe not. Given that virtually every teacher 
education program that applies for accreditation gets it, the logical 
conclusion is that teacher education programs must be universally 
“good.” With all the bad press teacher education programs have 
received in the past two decades, a program’s accreditation status has 
low credibility. We need to take transparency to a new level, and - 
like other countries do - make professional judgments public: not just 
whether the program is accredited or not, but also what accreditors 
have discovered as deficiencies, and what the program is doing about 
them. 

The Council for Higher Education Accreditation has taken a step 
in the right direction. As posted on the TEAC website: 

TEAC has revised its policy on public performance disclosure 
(Policy XI) in response to a newly instituted standard of the 
Council for Higher Education Accreditation demanding 
that accreditors require the programs and institutions they 


accredit to report program performance data. TEAC requests 
that institutions post a link to the Summary of the Case 
(now on the TEAC website at www.teac.org/membership/ 
teac-members/) on their website along with links to such 
other performance assessment summaries as they determine 
appropriate (including websites reporting Title II data). 
(Posted November 29, 2011, emphasis added) 

I would go even further than this. I would make public the 
recommendations of the TEAC panel and decisions of the Accreditation 
Committee of the Board, including any stipulations or weaknesses and 
the rationale for these, now available to the public only if the program 
chooses to release them. 

Taking this step would not resolve any of the five tensions I’ve 
written about in this chapter, as they represent dialectics that cannot 
and should not be resolved. Making TEAC’s professional judgments 
public would however make these tensions clearer and more vibrant. 
First, it would continue to honor TEAC’s commitment to inside-out 
assessment, while acknowledging the need for more than just token 
outside-in perspectives. Second, it would recognize a program’s 
uniqueness, while affirming that some standards are not negotiable. 
Third, it would make public the reasons for TEAC’s concerns about 
a program, and force auditors, panelists, and committee members 
to reveal their biases about “what counts” as evidence. Fourth, by 
isolating weaknesses in both a program’s quality control system and 
its student learning evidence, it would underscore the importance of 
both process and outcome. Fifth and finally, it would maintain TEAC’s 
firm commitment to program improvement while also putting some 
teeth into public accountability. 
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Doing Better: 

Musings on Teacher Education, 
Accountability, and Evidence 


— Suzanne M. Wilson 


Healthy fields have some humility about current approaches 
and theories and maintain an open mind in their consideration 
of the alternate approaches that are advanced by colleagues. 
They suspend judgment until the evidence is in. They balance 
their legitimate skepticism with generous assumptions about 
the motives of others. . . . (Murray, 2005, p. 315) 

We live in an age of calls for increasing accountability, whether 
on Wall Street or in school districts. In teacher education, this 
accountability takes several forms. Program accreditation has been 
around since at least the 1950s and the National Defense Education 
Act; NCLB’s Title II placed myriad requirements on higher education 
and state departments of education for testing teachers; currently there 
are calls for value-added assessments of teacher education programs. 
While some teacher educators prefer to ignore these increasingly 
insistent calls for accountability, we live in an age in which alternatives 
to university-based teacher education proliferate. While alternative 
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pathways into teaching do not prepare the majority of teachers, they 
represent an unprecedented threat to what some consider a monopoly 
on teacher preparation held by higher education. How we respond to 
demands to demonstrate our worth matters. 

Of course, accountability, in and of itself, is not a bad thing. 
We hold banks accountable for being good stewards of our money. 
Parents regularly teach their children to be accountable for their 
actions. We hold doctors, lawyers, architects, and nurses accountable 
to their respective professional standards for practice. In this sense, 
the heightened call for accountability in teacher education and in 
the programs that prepare teachers is reasonable, even responsible. 
Demonstrating that our programs meet professional expectations 
is part and parcel of claiming membership in any guild. And just 
as a parent does not accept a child’s protestations that she means 
to be responsible, so too it is not unreasonable that the public and 
policymakers want teacher education programs to produce something 
other than promises of good intentions. Instead of only documenting 
inputs, the argument goes, programs should also demonstrate 
outputs and effects. Data has become the coin of the realm, whether 
in evidence-based medicine or data-driven decision-making. Data will 
make us leaner, more efficient, more honest. Looking at the data will 
help us become better versions of ourselves. 

In addition to this emphasis on evidence and data, we are 
experiencing a shift in what the public considers the appropriate unit of 
analysis. Former accountability paradigms held students accountable 
for their learning; current discourse focuses intently on the teacher 
(through value-added measures) and schools (through publicly 


available school report cards). Similarly, if teachers did not perform 
well, old paradigms pointed to their background; if teaching was not 
of high quality, it might be due to who enters the profession. That logic 
no longer dominates. Instead, the teacher preparation program - its 
selection criteria, its content, its resources - is what matters. 

Of course, these calls for accountability and evidence have not 
been without their critics. “Being held accountable” can communicate 
a sense of distrust or suspicion. Being held accountable can also 
misplace attribution; assuming that one has control over factors 
and forces that are beyond one’s reach. Accountability can also be 
operationalized to mean standardization and/or quantification. 
Accountability can overemphasize that which is measurable and 
marginalize equally important aspects of our lives that prove less 
amenable to quantification. Thus, it can distort rather than enhance, 
constrain rather than enable. 

So how do we as teacher educators act responsibly in light of 
increased calls to be accountable and a heightened awareness that 
such calls are neither innocent nor innocuous? How we answer this 
question, I believe, will fundamentally shape how we are positioned 
in present and future discussions about who should prepare teachers, 
where that preparation should take place, and what the focus of 
preparation ought to be. My own answer? We can do better. 

****** 

In the spirit of “do as I say,” let me start with a story from my own 
university. In terms of data, we swim in a sea of it. We have information 
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on every undergraduate before and after matriculation. We know their 
ACT scores and subscores, the high schools they attended and their 
class rank. We know their gender, race, ethnicity, high school GPA, 
and high school grades in specific courses. We know what advanced 
placement courses they took and their scores. After they arrive at 
MSU, we know the college courses they have taken, who taught them, 
how large the classes were, and what grades they received in each 
class, as well as their GPA in their majors, minors, and professional 
coursework. 

That’s not all. During our program’s yearlong internship, we have 
field instructors observe each intern about to times; in addition, both 
cooperating teachers and field instructors fill out mid-year and end- 
of-year evaluations. We also get data from the Michigan Department 
of Education (MDE) every year on the performance of our prospective 
teachers on content examinations, as well as feedback on MDE surveys 
taken by all of our graduates and their field instructors. 

We’re awash with data: test scores, calls from parents, letters 
from students, emails from principals and teachers, students’ GPAs, 
interns’ mid-year evaluations. Every day we respond to information 
we have received and interpreted: A doctoral student has to be 
removed from field instruction because he is not visiting an intern’s 
classroom regularly enough and his feedback is neither frequent nor 
constructive. An instructor needs support in learning to teach her 
class better, her students are confused and feel like they are wasting 
their time. An intern needs to be moved to a different classroom 
because her collaborating teacher is not providing her with sufficient 
opportunities to teach. The state has changed the requirements for 


elementary teacher certification and testing, and new agreements 
need to be created across campus with the general education and 
disciplinary department faculty. 

As we respond, we’re constantly using data. We evaluate the records 
of all of our current students and make sure that they are abiding by 
requirements concerning their grades in their professional courses 
and other required courses across the university. We do background 
checks before they enter the internship, and we have a well-articulated 
set of expectations for what they have to do in order to progress to 
that internship. As Department chair, I read the student evaluations 
(SIRS) for every course for every instructor for every term, as do the 
instructors for each class. Every faculty member who supervises a 
course with multiple sections also reads and reviews the SIRS for all 
instructors for whom they have supervisory responsibilities. 

We submit reports annually to the Teacher Education 
Accreditation Council (TEAC) as part of our accreditation agreement, 
for which we conduct analyses of our students’ grades in their content 
and professional classes, as well as review other relevant evidence. 
We regularly write reports to the MDE concerning changes in state 
expectations and policies, whether reporting on experimental 
programs or innovations in using technology in teacher education 
or summarizing feedback we receive in our state-mandated survey 
of school principals. In addition, many faculty and doctoral students 
regularly conduct research on our teacher preparation program, 
observing classes, shadowing interns, gathering qualitative and 
quantitative data concerning the program’s content and character, as 
well as its effects on participating teachers. Dissertations get written, 
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articles and books are published. 


Our program and our faculty are, thus, in perpetual motion, 
and almost every activity is either catalyzed or informed by data and 
evidence. Accountability is everywhere: students are accountable 
to the program, instructors and held instructors are accountable to 
their supervisors, everyone is accountable to the chair, the chair is 
accountable to the dean, the college and the program are accountable 
to the university and to the state. 

******* 

Given how accountability and data have seeped into the marrow 
of our organizational bones, calls for additional accountability can be 
experienced as both tiresome and suspect. At the very least, calls for 
evidence take time and other resources (personnel and the technology 
and infrastructure needed for data collection, management, and 
analysis). Most recently this came in the form of producing documents 
for the National Council of Teacher Quality’s U.S. News and World 


' All of this takes place on a much larger stage, for teacher education is not the only 
work we are required to do in my department. The university expects us to offer 
courses with high enrollments and attractive majors and minors and to have thriving 
and rigorous graduate programs at the masters and doctoral level, which have high 
application and low admission rates. In addition, faculty are expected to raise grant 
money and publish research articles, serve on important professional committees, 
and do outreach to local and global communities and constituencies. To evaluate our 
productivity and to keep us accountable, the university has a range of metrics they 
use: student enrollments, class size, student retention and graduation rates; the GRE 
scores of our admitted doctoral students; research dollars raised per faculty member 
per year and over three years, the number of publications per faculty member, awards 
won by faculty and their doctoral students; the diversity of the student, staff, and 
faculty; retention rates for faculty of color; and the like. Administrators and faculty are 
fed this information every year, and we are all expected to respond with specific goals 
and quantified targets that will demonstrate how we are striving to improve. 


Report review, for which we had to retrieve syllabi for all required 
courses (and every section) that prospective elementary teachers took 
in one year (totaling 402 syllabi), as well as the guidance documents 
and written policies used for our communication with collaborating 
teachers, school principals, field instructors, and everyone else 
involved in field experiences (97 documents). All told, for one round of 
data collection, three people worked full time for a week, two additional 
for three days, and 100s of emails were sent and responded to. (This 
is on top of the ordinary work that needs attending to.) To be worth 
those costs, we have some obligation to use calls for accountability 
to our advantage, as opportunities to examine ourselves, while 
also proactively fighting the tidal pull toward standardization and 
quantification that can constrain and limit our efforts to responsibly 
educate. So as I consider what teacher educators ought to do in the 
face of this press to be accountable, again I can only ground it in what 
I think we might do in our own program, and how I think we can 
improve. 

Such improvement would mean attending to both the individual 
prospective teacher/instructor and the program. Recall how much 
information gets collected and used for making individual student 
decisions. In the internship year itself, we have 500 students, 10 
observations, 2 evaluations, which means that we have at least 
6000 data points on the program’s effectiveness. We also have the 
information we gather on their grades and grade point averages, used 
both to admit them to the program and to monitor their progress, 
which includes their performance in liberal studies, disciplinary 
majors and minors, and professional coursework. We’re quite vigilant 
in watching our students’ progress, an expensive enterprise. 
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We place these data - the interns’ grades and GPAs, our 
observations and their evaluations - in individual folders, filed away 
in case of a grievance . 2 The individual is our unit of analysis. This 
makes sense: after all, in our daily work our obligation is to make sure 
that every prospective teacher masters the content of the program 
and is a successfully launched beginner who we recommend for initial 
state licensure. 

Save for the data analyses we do for our annual reports to TEAC for 
accreditation, we seldom examine or use these data to have discussions 
among ourselves - or with our colleagues across campus - about what 
aspects of the program are stronger and weaker. We don’t examine 
correlations between the background characteristics of our teacher 
education students and their later teaching performance (including 
their ACT scores, high school rank, high school classification, grades 
and GPAs in disciplinary or general studies courses). We don’t use 
the observational data that we gather to track interns’ progress to 
assess program effectiveness by asking, “What patterns do we see in 
what our interns can and cannot do?” Further, although we carefully 
review the student evaluations for signs that individual instructors 
need support in improving their instruction, we do not systematically 
and holistically analyze the sum of the coursework and its correlations 
to interns’ performance in their fifth year. We are focused almost 
exclusively on the students, or on the individual instructor. Without 
even considering gathering additional (or better) data, we have not 


2 This was brought home to me one day when an intern filed a grievance about being 
removed from the program midyear, and when I went to examine her file, I was told 
that all of the materials were at the home of the field instructor who worked with the 
intern. To get the materials, the field instructor had to drive an hour and a half to 
deliver them to my office. 


begun to plumb the potential of the data we swim in to explore our 
program’s strengths and weaknesses. 

******** 

Shifting our vision to become bifocal - with one eye on the 
prospective teacher and another eye on the program - might seem 
straightforward. New technologies make it possible to put a tablet 
in the hands of every field instructor and collaborating teacher so 
that those observations and mid-year evaluations are electronically 
deposited in a centralized warehouse of information, and easily 
accessed by appropriate and appropriately trained users. Other 
technologies now make it possible that materials digitally uploaded 
into course management systems - students’ writing, course syllabi, 
student evaluations, tests, videotapes of interns’ teaching - could 
also be deposited in a program’s electronic warehouse and viewed 
by people other than the individual instructor or student. Other 
technologies could easily de-identify materials so that individual 
students’ privacy would be protected. Faculty meetings could involve 
discussing the firsthand evidence available to all program leaders, 
viewing and discussing interns’ teaching, examining student work. 
Course instructors could see evidence of what happens with students 
in other courses and later in the program, and have a longitudinal 
view of how their course fits into a more collective enterprise. 

Notwithstanding the need to be vigilant in protecting the rights 
of individual students’ privacy, limiting who viewed any data that 
was not de-identified, the possibilities for seeing a bigger picture of 
what a program is doing or not doing are mind-blowing. When I first 
realized that we had 6000 data points on our program in the form 
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of field observations and evaluations, for example, I was gleeful. The 
interns are in the program for three years, at any one point in time, we 
have 1500 teachers in the system who go through in cohorts. We have 
access to a wide array of other information through the university. As 
a researcher, I would give my eyeteeth for that kind of qualitative/ 
quantitative data, especially in the field of teacher education where 
there has been historically so little funding, an obstacle for large-scale 
studies of program effects. 


********** 

Others were less excited. When I proposed moving note-taking 
from the individualized forms to a computer app that would make it 
possible for field instructors and collaborating teachers to log on, type 
in their notes, and press a button to have it sent to the department, 
I was told that the field instructors had abandoned the observation 
form (they felt it constrained them) and that the different subject 
areas in secondary teacher preparation could not agree on a common 
form (the subject matter differences are too great to allow for such 
uniformity). I also found out about some field instructors who had 
never submitted any information and the considerable amount of 
missing data that were not accounted for. 3 Further dismaying was the 
discovery that a number of doctoral students and fixed term faculty 
did not use the syllabi or assignments that were carefully crafted by 
the course coordinator, choosing instead to create a course that they 
felt was better suited for their students. This resonated with our 
experiences when attempting to use program wide assessments as part 


3 This was reinforced when we conducted an audit of our files for the purposes of our 
TEAC review; for a randomly selected group of students, we found many files with 
missing information from throughout their program. 


of our accreditation process: in numerous courses, instructors either 
did not use the standard assessments, or fundamentally altered them, 
and hardly anyone used the common rubrics that had been created for 
scoring. 

The strong resistance represented in these actions is not powered 
by a desire to be sloppy teacher educators; in fact, many instructors 
resist what they consider a march toward standardization in the name 
of maintaining quality and personal attention. Of course, in a program 
as large as ours, the chances of hiring an irresponsible field instructor 
who fails to fulfill her obligations, or a responsible one who forgets to 
bring the papers to campus increase. That the field instructors found 
the observation forms constraining was not unsurprising either. 
We already know that field instructors and collaborating teachers 
often feel a tension between their dual role of both supporting and 
evaluating new teachers. Furthermore, our program is constantly 
churning and we encourage students and faculty to be critical of 
the courses and tools that we use. It is also not surprising that some 
instructors throw out a common syllabus or assignments in favor of 
what they want to teach; closing the classroom door and doing what 
one pleases has a long history in American schooling, at both the K-12 
and higher education levels. In higher education, instructors regularly 
invoke the language of “academic freedom” in defense of this choice. 
It is not always a bad idea; the fact that a syllabus is standardized does 
not make it good. 

While some might want to simply dismiss this resistance as 
evidence of the low quality of teacher education programs and faculty, 
there is considerable evidence higher education in general lacks 
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what some have called a “culture of evidence.” Understanding this 
resistance to making one’s work public and increasing internal and 
external accountability as rooted in cultural norms is an important 
step in finding ways to fight it. And to learn from it. Making sure 
paperwork is filed is relatively straightforward. But shaking the deep 
roots of this widespread resistance to collective and public inquiry into 
the quality of our programs is much harder. And it should be. We want 
to make sure that any time we increase accountability, that change 
is intentionally designed to improve, not constrain, program quality. 
This too we can do better. 


******* 

First off, we can insist on a generous conception of evidence. The 
evidence-based movement has, in general, elevated quantifiable data 
and experimental designs over narrative and observational data, and 
this had lead some teacher educators to reject calls for accountability 
based on the assumption that the only evidence that will be taken 
seriously is that which is quantifiable. But a narrow view of evidence is 
not a requirement of a commitment to evidence; in fact, many in the 
evidence-based medicine movement argue against the oversimplified 
interpretation that evidence only refers to data produced from 
experimental designs. We can choose to conceive of evidence in a broad 
way, and one that does not presume a hierarchy of evidence (with 
randomized, double blind experiments at the top) but instead insists 
on defensible alignment of the evidence best suited to the question 
at hand. For some questions, one needs carefully done observational 
work. For other questions, one needs large-scale experiments. In 
teacher education, we need careful observations of our interns 
working in the classrooms of collaborating teachers, and we also need 


results of their performance on standardized tests. Ideally, we would 
have multiple methods of collecting data for each major outcome we 
care about so that we might triangulate across those different sources 
of inherently limited empirical evidence. To use my own example, the 
6000 observations that we have of our interns in one given year are a 
mix of some Likert-scale ratings with a lot of observational notes and 
narratives, peppered with examples drawn from those observations. 
Similarly, instructors in courses write comments on papers, as well 
as assign grades. As part of the normal work of teacher education, we 
regularly produce evidence that is both qualitative and quantitative. 
There is no reason why we cannot use the full range of evidence we 
have available. 

Well, there is one reason. We should only use evidence we trust, 
and we could do better in terms of increasing the quality of the data 
we produce. Whether one is most comfortable conceptualizing these 
issues as with the language of reliability and validity or the language 
of credibility, transferability, dependability, conformability, and 
soundness, we have an obligation to ensure that the data we use to 
make decisions both about individual prospective teachers and about 
the program’s quality are of high quality. 

Increasing the quality of our evidence could take many forms. 
University professors and K-12 teachers alike create assessments and 
tests of what students have learned that are never themselves assessed 
to learn whether they accurately measure what they are designed to 
measure. In the 34 years that I have been a teacher, no one has ever 
asked me to prove that my tests are valid assessments of my students’ 
learning. Our field instructors get considerable support in learning to 
do the work; they participate in a mandatory seminar about learning to 
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teach, and they meet regularly with supervising faculty. But we seldom 
engage in exercises to make sure that their observations are reliable. 
We don’t check the inter-rater reliability of their observations; we do 
not conduct validity studies of their evaluations and reports. Similarly, 
if we use common assignments across course sections, we do not 
examine whether the grading is comparable by calculating inter-rater 
reliability within a course. 

Without demanding that all faculty turn their attention to 
producing technical reports of the reliability of their measures, we could 
do better. We could look at the consistency of a student’s performance 
within and across classes; we could analyze the variability in grades 
across different sections of the same course in courses across the 
university. We could do outlier analyses of students who fail courses 
or the internship; we could systematically rotate field instructors 
to check the consistency of observational data. We could adopt the 
practice of many university writing programs where instructors use 
common end-of-course assignments, and another instructor grades 
the papers from their seminar as an internal accountability check on 
consistent instruction and expectations. 

Of course, studying our measures and assessment tools would 
surely show us that our measures are limited. In discussions of why 
ratings vary on an observational instrument, one might work to 
calibrate the raters so that their scoring is more similar. Alternatively, 
differences might lead to reconsidering the instrument itself, and 
altering it in ways that are more responsive to what the raters need to 
record. Examining a test in a disciplinary course might raise questions 
about whether the test is testing the intended learning. At Michigan 


State, the instructors of our general education courses (called Integrated 
Studies) decided to do that. When they examined the assessments used 
across the integrated studies courses, they were chagrined to discover 
that the content of the tests did not align with the goals of the courses. 
They were also surprised to learn, in consultation with testing experts 
on campus, that if one wants to test a student’s knowledge of a concept 
- say, global warming - there are mathematical ways to estimate how 
many questions one needs to build a scale comprised of responses to 
individual items. Their common practice - for one idea, generate one 
or two questions - was problematic if they wanted accurate measures 
of what students learned. The assessments that the group created after 
working collectively on their tests and having them critically appraised 
by outsiders were much different than what they had first created. 

*********** 

I use the example of the Integrated Studies faculty intentionally. 
While a generous view of evidence and better instruments might 
well contribute to improvements in our programs, one cannot 
underestimate the need for joint work. 

We work within cultures, and those cultures have norms and 
structures that pull us in certain directions. Without making any 
generalizations about teacher education more generally, I will assert 
that our culture is one of a focus on the individual student and teacher. 
We work hard to select the right students, to track their progress, to 
challenge them with good courses, to document their learning to teach 
in meaningful ways, all in the name of being confident that anyone 
we recommend for state certification is a well-launched beginner. As 
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part of that work, we, of course, work on the content and character of 
the program: we are constantly changing readings and assignments 
in courses, adapting both to new knowledge in our relevant fields and 
to our experiences with what works and does not work with students. 
However, the orientation is largely focused on making sure that 
individual students make it through the program or are counseled out. 
Everything we do, our daily actions and reactions, all reinforce that 
orientation. 

Shifting those norms to include a collective and programmatic 
perspective meets with considerable resistance, both voiced and 
behavioral. When I suggested that we create an electronic warehouse 
of data so that we might both improve program quality and make it 
possible for doctoral students and faculty to conduct research, one 
faculty member told me, “You’re acting like Big Brother,” invoking 
Orwell’s omnipresent dictator. Others, less willing to publically raise 
concerns, simply close their classroom doors and do what they think 
is best. Declaring that we can do better will not change our orientation 
toward the individual, nor will insisting on using standard forms lead 
to high quality evidence. Words alone cannot fight those institutional 
structures and cultural norms. 

This is, by far, the most challenging aspect of participating in 
the TEAC accreditation. When visiting campuses, auditors listen for 
language that suggests that the entire institution has shifted from 
doing program analyses for the purposes of submitting accreditation 
reports (“for TEAC”) to doing program analyses for the purposes 
of informing the program’s on-going improvement. But creating a 
“culture of evidence” takes years, and it does not happen by directive 
from an administrator or a state department official. It is especially 


hard in a political and intellectual environment in which some calls for 
accountability are of questionable motive and in which K-12 teachers 
and (increasingly) faculty in higher education feel pummeled by 
criticisms and calls for writing more reports. 

Joint work, authored by the insiders, can help. The Integrated 
Studies faculty started out with modest goals: to do some internal 
monitoring of the content of their classes. As serious teachers, however, 
they were pulled more and more into the activity as they began to 
uncover evidence that challenged their assumptions. (None of this was 
motivated by accreditation requirements or state-mandated reviews.) 
And the more they discovered, the more interested they became in 
gathering more information and reaching out to experts who might 
expand their own understanding and capacity. This is not surprising; 
as scholars, they regularly interrogate colleagues’ work with respect 
to the use of high quality data, reasoned arguments, and rigorous 
analysis. When their inquiry into their own teaching tapped into their 
scholarly habits, the work they were able to do both challenged them 
and led to critical examination and considerable curricular change. 
With their imaginations fired and driven by their own curiosity about 
work they were seriously invested in, they energetically opened up to 
changing how and what they taught. 

******** 

Earlier I mentioned that the call for accountability and the 
increased use of evidence is neither innocent nor innocuous. Nor is 
evidence - as generously conceived as possible - the only basis for 
making decisions about education. It might help us a lot to look 
more carefully at information that we already gather, and to strive to 
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improve the quality of the evidence and our interpretations, we need 
to also always ask the hermeneutical question, “In whose interest are 
these data collected and interpreted? And to what end?” Our critics are 
quick to point out that it is in our self-interest that we collect much of 
the data we do in teacher education programs (personally, I am never 
persuaded that high GPAs in professional coursework is a trustworthy 
indicator of teacher learning; I’ve reviewed the GPAs in courses across 
the university and grade inflation is as omnipresent as the press for 
being more accountable). And we’re equally swift in pointing out that 
some of our critics’ self-interests are questionable as well. Here too we 
can do better. 

Internally, we can begin considering how our own self-interests 
seep into our assessments of our students and our programs. 
Externally, we can commit to raising the quality of the debate about 
the trustworthiness of teacher education and the need for increased 
accountability by resisting oversimplifications that turn a request for 
evidence into a hegemony of numbers or a genuine interest in what we 
might learn from programmatic views of our students’ learning into 
Orwellian dictators robbing all citizens of their privacy. There is no 
honor in that. 

That is not to say that we ought not be wary. In the end, high 
quality teacher preparation needs to be informed by the values and 
beliefs of multiple stakeholders - the public’s views of who should 
teach their children, the disciplines’ views of what it means to be well 
educated in a subject area, educators’ views of what it takes to be a 
good teacher. Teacher preparation needs to also aggressively snatch 
up any and all new knowledge generated by high quality research on 
teaching and learning, no matter what the tradition the researchers 


follow. We need to resist the imperialism of some who would narrow 
the views of what counts as good teaching or good evidence, and we 
need to nurture a culture of open and critical debate about what it 
takes to prepare people to teach. We’ll have no credibility in those 
debates if we do not model how to look critically at ourselves as well, 
in healthy and generative ways (which includes questioning whether 
responding to a call for more accountability is always a good thing). 
And that takes, as Murray (2010) argued, humility, generosity, and 
skepticism. It also takes time. 


57 



Higher Education, Accreditation, 
and Change, Change, Change: 
What’s Teacher Education To Do? 


— Barbara Brittingham 


This paper starts with the observation that accreditation is 
more visible than ever because higher education has become more 
important and more expensive to individuals and society. From that 
observation, I will focus on three developments that are sharpening 
the focus of accreditation and can increase its power to help us 
understand educational effectiveness, and then provide some advice 
to institutions on how they might respond to these new developments. 
The paper will conclude with a brief discussion of implications for 
teacher education. 

Accreditation is more visible than ever because higher 
education has become more important and more expensive 
to individuals and society. 


Accreditation, that once sleepy enterprise cresting the university’s 
horizon every ten years, has suddenly become visible. If not center 
stage - and accreditation is, after all, a supporting player - it is at 
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least on the stage and no longer in the restricted view seats. No longer 
is institutional accreditation a decennial event; it has developed into a 
relationship. And while they may not be pleased about it, no longer are 
presidents surprised at the number of specialized accreditors engaged 
with their campus. 

Why? What has happened to bring accreditation more visibility? 
Accreditation is an American invention that traces its roots to 1885 
with the founding of the New England Association of Schools and 
Colleges. Followed by similar organizations in other parts of the U.S., 
these now-regional accrediting associations developed slowly over 
several decades. Since the 1950’s regional accreditation has operated in 
a format recognizable today, with the common elements of standards 
to be met, an institutional self-study, a visit to validate the self-study, 
and a decision by a group of peers. Also since the 1950’s, the federal 
government has relied on its system to ‘recognize’ accreditors as 
‘reliable authorities’ on the quality of education for purposes of access 
to federal financial aid. 

The recent increase in the visibility and importance of accreditation 
is due to at least four factors: 

1. an economy that increasingly depends on a college-educated 
workforce, and the importance of a college education for 
individuals’ success in the economy; 

2. more students going to college, both of traditional age and 
adult students; 

3. lower international rankings of U.S. higher education; 


4. a dramatic change in how higher education is funded: less 
state money, more federal money, and higher student debt . 1 

Thus, higher education in the United States is more important 
and more expensive than ever, and the anxiety of individuals about 
cost and benefit is exacerbated by the collective anxiety that our 
system of higher education is not as effective as it should be, at a time 
of considerable economic uncertainty. In the popular press, the plight 
of two groups of students has been particularly prominent: students at 
for-profit colleges, an institutional type that has concerned some in the 
federal government for the greatly increased amount of financial aid 
these institutions consume; and students at law-schools. Both groups 
of students, who otherwise have little in common, have been portrayed 
as casualties in a system that does not focus on skills employers want 
most nor reliably produce employment at sufficiently high salaries for 
these graduates to repay their often considerable student loan debt. 

In the search for solutions to the problems of high importance, 
high cost, high debt, and uncertain prospects for employment 
sufficient to repay the loans, the summative question for individuals 
and for policy makers is, What are we getting for our money? And 
the hook that policy makers often reach for is accreditation. Because 
of the link with federal financial aid, accreditation represents one 
of the few avenues of direct influence - i.e., regulation - that the 
federal government can exercise over colleges and universities. While 
this increased prominence is a mixed blessing for accreditation, it 


' Funding for Pell Grants alone has nearly tripled since 2008 to over $40 billion. 
Student loan debt approaches $1 trillion, and the average debt among baccalaureate 
graduates with debt is over $25,000. 
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does give accreditation more importance, and therefore changes to 
accreditation have a heightened impact on institutions and programs. 

Three developments are sharpening the focus of 
accreditation and can increase its power to understand and 
improve educational effectiveness. 

Accreditation is becoming more focused on data and 
evidence. As accreditation has become more important, it has also 
increasingly focused on data and evidence. Some of this increased 
focus is due to political pressure (see above). And some of the 
increased focus is a natural by-product of institutions increasingly 
having the tools that allow an evidence-based approach to self-study. 
Institutional research offices, student record systems, and other 
technologies have provided institutions with greater capacity to use 
trustworthy evidence and data for self-study. 

Some of the heightened expectations are due to what accreditors 
now expect to see in institutional reports. TEAC is a prime example 
here, with the institutional report (aka self-study, or Inquiry Brief, in 
the TEAC system) being a program’s own audit that traces its policies 
to their implementation in practice and examines evidence of student 
learning. Also, in the TEAC system, the visit step of the process is 
conducted by trained auditors rather than a team of (arguably) lightly 
trained peers, stripping away the theatre associated with normal visits 
and allowing the process a greater focus on verifiable evidence and 
data. 

TEAC takes the general emphasis on evidence and data one 
step further by stating that its process is “inquiry driven, starting 


from the faculty’s own questions and curiosity about the program’s 
accomplishments,” 2 a further step in moving accreditation from a 
compliance step to an exercise with scholarly implications. And in fact 
“faculty learning and inquiry” is the second of three of TEAC’s Quality 
Principles upon which its accreditation is based. 

Thus, with accreditation’s increased visibility locally and nationally 
and increased institutional capacity to gather and use meaningful 
evidence and data, accreditation is, arguably at least, poised to become 
increasingly useful as a stronger means of quality assurance as well as 
quality improvement. 

An increased focus on student success is connecting K-12 and 
higher education. The history of connecting secondary and higher 
education is one of good intentions, great effort, and less-than-stellar 
success. The current large-scale effort centers on the Common Core 
State Standards, now adopted by all but five states. PARCC - the 
Partnership for Readiness for College and Careers - is a 25-state effort 
funded by $186 million federal grant to develop a “next generation” 
K-12 assessment system centered on English and math skills needed 
for success in college. The Lumina Foundation is also funding state 
efforts to connect the success of students in secondary education with 
their success in higher education, and in some localities there are 
individual efforts that bring these principles close to the ground. 3 


2 http://www.teac.org/wp-content/uploads/2009/03/TEAC-StandardQuality.pdf on 
12.30.11 

3 for example, see http://www.neasc.org/ 
downloads/201 1_annual_meeting_presentations_CIHE/ 
NEASCCommonCorePanel4FlemingCummingsMarkMarkPeaseSakoffs.pdf 
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There are at least two outcomes to be avoided here: 1) measures 
of high school achievement that don’t predict college success; 
and 2) inducing Campbell’s law by creating a high-stakes test that 
becomes corrupted by its very importance. In the case of the former, 
Massachusetts’ community colleges have found that while average 
student scores have increased on MCAS, the state’s high stakes high 
school-leaving examination, the proportion of community college 
students requiring development math and writing courses has 
remained essentially flat. In the second case, the cheating scandal 
in the Atlanta schools (uncovered in July 2011), for example, makes 
Campbell’s point. 

Agreeing on useful measures of success in education is notoriously 
challenging. For example, the most common measures of success 
in higher education - at least in the public policy debates - are the 
IPEDS retention and graduation rates. But because these rates 
include only first-time full-time students, ignore transfer students, 
and have an upper limit of 150% of the estimated time-to-degree, they 
are of limited utility. For some institutions with high rates of part- 
time and transfer-in students, the IPEDS rates may measure retention 
and graduation rates for only about 10% of the student body. And 
- no surprise - students from more advantaged backgrounds tend 
to persist and graduate at higher rates than students from more 
challenging backgrounds. 

Help is on the way. Some states are developing student record 
systems that can track students from K-12 through higher education. 
And the American Association of Community Colleges, for which 
members’ traditional retention and graduation rates are most 
problematic, has developed a Voluntary Framework of Accountability 


that will help community colleges use more inclusive and descriptive 
measures of retention and graduation. Also, the National Student 
Clearinghouse now includes virtually all public and non-profit 
institutions and a growing proportion of for-profit institutions; 
participation in the Clearinghouse lets colleges and universities learn 
the destination and success of students who transfer out and can also 
help them track the destination and field of study for students who go 
on to a higher degree. 

In teacher education, there are now efforts to measure the success of 
teacher education programs by looking at how much pupils learn from 
their teachers-in-training, or from the graduates as newly prepared 
teachers. For example, Louisiana now has a student record system that 
incorporates K-12 students in public schools and teacher education 
students from programs in public and private higher education as well 
as alternative programs and is gaining some experience in its use. The 
measurement challenges here are considerable, and it is too early to 
know under what conditions these ambitious systems can produce 
results that are sufficiently reliable to make judgments on program 
success. In addition to measurement problems, other challenges are 
also present: Will such systems further narrow teacher preparation, 
with new teachers being prepared to teach those topics that will be on 
the school assessment system? That would be a high price to pay for a 
system that purports to measure the effectiveness of teacher education 
programs. 

Technology and open-access education are changing our 
understanding of higher education. If the past twenty years 
have seen the introduction of technology to deliver instruction in the 
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classroom - physical or virtual - we are now seeing the introduction of 
technology to expand the boundaries of the university and to harness 
the back end of technology to understand human learning. 

Consider three new initiatives: 

l. MIT’s OpenCourseWare project was announced in 2002 
as an effort to put all of MIT’s course-related materials on-line 
and freely available. By November 2011, over 2,000 courses 
had materials available on-line, including discussion topics, 
problem sets, answers to the problems, streaming video 
lectures, and interactive web demonstrations. By making 
course material freely available, MIT extended the benefits 
not only to students and the public, but also to faculty from 
institutions around the world who can benefit from seeing the 
curriculum and expectations of MIT for a particular course. 

In December 2011, MIT took the project to the next step: 
in spring semester 2012, it will begin to make interactive 
versions of courses available on-line through an initiative 
known as MITx. Two major steps are of interest here: (1) 
students will have an opportunity to demonstrate mastery 
of the subject matter and by so doing can earn a certificate 
from MITx; and (2) the initiative will be accompanied by an 
Institute-wide initiative on on-line teaching and learning . 4 


4 MITx is a particularly interesting initiative from an institution that offers no degrees 
on-line and does not have an education school. The initiative comes from MIT’s 
deep commitment to sharing knowledge freely, its depth in technology and cognitive 
sciences, and its willingness to take some risks. 


2. The Kahn Academy, outgrowth of a Bangladeshi- 
American’s efforts to tutor his young cousins, now makes 
over 2,600 videos available on YouTube on topics that range 
from arithmetic to physics to finance to history. With over 
90 million lessons delivered, the non-profit organization 
announces its “goal of changing education for the better by 
providing a free world-class education to anyone anywhere.” 

3. The awarding of digital badges is another potential “game- 
changer,” as it has been called by Education Secretary Arne 
Duncan. The idea of digital badges is to create a system of 
validated learning that has occurred informally, i.e., outside 
the classroom. The learning could be validated by colleges or 
universities - or technical or trade groups, corporations, open 
courseware groups, or non-profit organizations. Recently, the 
John D. and Catherine T. MacArthur Foundation announced 
a $2 million competition for “leading organizations, learning 
and assessment specialists, designers and technologies to 
create and test badges and badge systems. . . .[to] unlock job, 
educational and civic opportunities; and open new pipelines 
to talent.” 5 Because the badges are digital, they can be linked 
to examples of student work, making them available for 
colleges and employers to review and consider. 

These initiatives, and others like them, hold extraordinary 
potential to open higher education - for institutions to learn from 
each other, to increase the useful ability of institutions and faculty 


5 http://www.macfound.org/site/c.lkLXJ8MQKrH/b.4196225/apps/s/content. 
asp?ct=1 1221 065 
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members to focus on learning outcomes, and to create efficiencies in 
education. Each of the above do or can make increased use of learning 
analytics that hold great promise to expand our understanding of how 
students learn. 6 For accreditation, these non-institutional, validated- 
to-credit sources of student learning increase the focus on outcomes. 

These changes in higher education and accreditation 
have implications for institutions and programs. Higher 
education is in a period of considerable and perhaps fundamental 
change. And therefore, accreditation will change as well. With an 
increased focus on accountability - quality assurance - accreditation 
is challenged to keep, and indeed strengthen, its focus on institutional 
and programmatic improvement. 

In a period of such rapid change, identifying the initiatives, 
investments, and risks - for institutions and for accreditors - is of 
key importance. Below are three suggestions for institutions and their 
programs: 

Build capacity for using evidence and data. Institutions 
can build the capacity of faculty to consider and use evidence 
and data in assessing student learning, judging program 
quality, and determining how well the institution and its 
programs meet its academic and social goals (e.g., teacher 
education graduates are employed at high rates by local 
schools; the institution is consistently on the list of 25 
that send a high number of graduates to the Peace Corps; 


See, for example, http://wp.nmc.org/horizon2011/sections/learning-analytics/ 


applicants to medical school are accepted at a high rate; 
graduates from business programs are employed quickly at a 
competitive salary). 

Building capacity of this sort is a significant undertaking, 
representing challenges that are academic as well as 
economic. On the academic front, specialized accreditation 
has provided much of the impetus to develop assessment 
systems at colleges and universities. Also, the methodology 
of assessment is essentially that of the social sciences; 
properly done, assessment calls on faculty to have a working 
knowledge of measurement, statistics, indicators - and 
qualitative methods that can provide a vivid description 
of student experience. So it is no accident that in the great 
majority of institutions, assessment of student learning has 
progressed more quickly in quantitative disciplines that 
have programmatic accreditation (engineering programs 
with ABET accreditation are the poster child here), and least 
quickly in the humanities (with no external accreditor and less 
tradition, interest, and capacity for quantitative analysis of 
student success). On the economic front, building the capacity 
to ensure that consideration of evidence and data become 
part of the departments’ routine is an added cost in the sense 
that it calls upon faculty to take on additional responsibilities 
at a time when many if not most institutions are feeling 
considerable financial strain. 

Building this capacity is not simple, and it takes time. 
Developing institutional habits of using data and evidence, in 
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the self-study for example, is subject to Mooers’ law, which 
states that people won’t use information if it takes more time 
to use it than to not use it. And after all, aren’t self-studies 
easier to write if we rely on description and assertion than if 
we have to take the time to round up the data and evidence, 
understand it, consider its limitations, discern the main 
points, and weave it into our narrative? Yes, they are. So the 
understanding of why evidence and data are important and 
useful is key to ensuring their use in the accreditation process. 

Many institutions have increased their capacity through 
offices of institutional research, assessment, and academic 
planning and centers for teaching and learning. Still, on 
the ground, at the department level, finding useful and 
methodologically respectable ways to understand student 
success in ways that can support program improvement 
represents a long-term undertaking. 

4. Study how students learn as well as assess what 
they learn. If assessment results only in information about 
what students have learned but not how they went about 
learning, the result may be very little useful information for 
improvement. This what but not how phenomenon is one 
of the great sources of frustrations with measures such as 
the Collegiate Learning Assessment which may - and the 
emphasis here is on may - provide information on how much 
students have gained from their college education - but that 
result in virtually no information useful to faculty or academic 
administrators that can be used to improve students’ 
education. 


Institutions have found numerous relatively easy ways of 
understanding how students learn. Results of standardized 
surveys (such as NSSE and CSSE) can provide some helpful 
information, along with comparison data. Curriculum 
mapping exercises can reveal how much opportunity students 
are given to learn important skills and concepts. And asking 
students directly how they go about a particular assignment or 
prepare for specific examination can be a useful, if sometimes 
humbling, source of feedback. Faculty development programs 
that focus on advances in cognitive science and educational 
psychology can bring in a more theoretical understanding 
of student learning. Finally, finding a way to use learning 
analytics can increase faculty sophistication about how 
students learn. 

5. Be open to, but validate, non-collegiate learning. 

MIT OpenCourseWare, digital badges: Higher education will 
increasingly need to consider when - or whether - to award 
credit for non-collegiate activity. For most institutions, it will 
be a matter of under what circumstances it will do so, not 
whether it will do so. 

To some extent, of course, this already happens. Bright high 
school students can take AP courses, pass a test, and the 
university can elect to award credit for students who pass 
at a designated level. Also, CLEP exams are designed to 
allow adult learners to demonstrate through examination 
that they have (somehow) learned what students learn in 
designated college courses. CAEL, the Council for Adult 
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and Experiential Learning, helps adults gain credit for what 
they’ve learned outside the classroom. And for over three 
decades the American Council on Education has maintained 
the College Credit Recommendation Service (CREDIT) that 
provides a structured and trustworthy opportunity for non- 
collegiate learning to be evaluated by academic peers, with 
recommendations for college credit that institutions are free 
to embrace or ignore. 

We can anticipate a surge in proponents of awarding credit 
for non-collegiate learning of various new types. The most 
common current example is StraighterLine. At $99 per 
month plus $39 per course, StraighterLine offers students the 
on-line opportunity to take a variety of common freshman 
classes, and over 20 partner colleges that take some or ‘many’ 
StraighterLine courses for credit. 

We can anticipate that once MITx is up and running, 
some institutions will grant credit for an MITx certificate 
demonstrating competency in, say, differential equations. The 
question here might well be: Why not? 

As with transfer credits, awarding AP credits, and credit for 
good scores on the CLEP exams, institutions are well advised 
to validate the award of credit. Do students who earn credit by 
or through another institution or organization demonstrate 
success in more advanced courses? 

Teacher education can illustrate how the current 

expectations for accreditation, the changes in the higher 


education landscape, and intentional actions on the part of 
institutions and programs can come together. 

Increased understanding of how students learn can pay double 
dividends in teacher education by increasing the content focus on how 
K-12 student learn as well as giving university faculty better tools to 
help improve programs for prospective teachers. Indeed, given the 
above conditions and trends, teacher education is uniquely positioned 
to demonstrate the importance of higher education programs having 
sufficient autonomy to make the enterprise of education and its quality 
assurance more thoughtful and more responsive to the public interest. 

Teacher education - that higher education enterprise most 
buffeted by external demands - might begin by conducting a self- 
examination in light of the above (and other) trends and forecasts. 
Recent major reform efforts in teacher education - and here it must 
be said that Frank Murray has been at the lead in all of them - have 
focused on key aspects of this most important enterprise. The Holmes 
Group sought to involve research universities, including their provosts 
and arts and sciences colleagues, in the serious business of preparing 
teachers; strengthened ties with schools, in the form of professional 
development schools, was at the heart of the reform. With Dan Fallon 
as the key partner, Project 30 focused on strengthening the links 
between arts and sciences and teacher education. And TEAC, an 
accreditation system based on an academic audit, has focused intensely 
on the use of evidence and data in making the public statement that 
a program is worthy of public trust because it is accredited. In all of 
these undertakings, Frank Murray knew that public trust in teacher 
education relied upon three things: 1) the commitment of research 
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universities; 2) partnerships with arts and sciences to assure teacher 
preparation in the subject matter; and 3) a publicly demonstrated 
reliance on data and evidence not traditionally associated with teacher 
education. 

There remains a category of data not typically well represented 
in teacher education programs, and as the above discussion may 
illustrate, the timing is right for this focus to emerge. A teacher 
education program focused more deliberatively on how people learn 
would represent something of a departure as programs that prepare 
teachers have traditionally focused more on the behavior of teachers 
than on helping prospective teachers understand how students learn. 

Teacher behavior, classroom setting, social conditions are all 
given greater attention in most teacher preparation programs than 
is the focus on individual or small groups of students engaged in 
the work (and joy) of learning. Many assessment systems focus on 
how well (prospective) teachers know their subject matter and their 
pedagogical skills, an approach that does not automatically include a 
deep understanding of developmental psychology, how students learn, 
human variation in learning, and content-specific ways of knowing. 
Indeed, the new interest in judging teacher education programs by 
how well pupils learn from newly prepared teachers, while sensible 
on one level, runs the risk of the unintended consequence of further 
narrowing teacher education programs so that they (only) prepare 
teachers who can help their students do well on standardized tests. 

Imagine an analogous emphasis in medical education: What 
results would we expect from training doctors by focusing on the 


desired behavior of physicians, health care regulations, the sociology 
of hospital settings, bedside manner, and arranging the doctor’s office 
- all without providing a solid foundation in the sciences and focusing 
on understanding the individual patient? Being a bit irreverent here, 
we might get Dr. Nick, perhaps, from The Simpsons, or Dr. Spaceman 
from 30 Rock. 

On a more serious note, to further illustrate the difference in 
approaches and emphases between medicine and teacher education, 
a Google search for “characteristics of effective doctors” yielded 
4 results, while a search for “characteristics of effective teachers” 
produced 194,000 results. This emphasis on the “characteristics of 
effective teachers” is a reflection of a teacher-centered and not learner- 
centered approach to teacher education. 

The question here for accreditors is whether institutions and 
programs have regular and effective means of helping faculty 
understand how their students learn. And for teacher education 
programs, whether those programs help their students - candidates 
to become teachers - understand how the children and young people 
they aspire to teach learn. The academy is gaining knowledge every day 
about how people learn, and it behooves teacher education programs 
to lead the way in incorporating this knowledge into their curriculum 
and assessment methods. 

Anyone who has heard Frank describe Piagetian experiments or 
who has seen one of his course syllabi can see that, once again, Frank 
is ahead of the game here. 7 


7 See, for example: http://www.udel.edu/fmurray/edst823.html 
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What Future for Accreditation: 
The Challenge and Opportunity of the 
Accreditation - Federal Government 
Relationship 


— Judith S. Eaton 


This paper addresses an emerging challenge to accreditation and 
its fundamental values of institutional autonomy, academic freedom 
and institutional mission. To the extent that the paper frames a 
major challenge, focuses on our core values and offers solutions, it 
reflects what many of us have gained from Frank Murray’s work 
and many contributions. I have learned much from Frank over the 
years, especially the importance of facing issues squarely, the need 
for resolve in the face of challenge, the importance of the values that 
drive our work and the urgency of courage to take action. 


Introduction 


A major influence reshaping the work of accreditation is its 
relationship with the federal government. In particular, government 
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actions during the past six years are significantly altering the role that 
accreditation plays in the society, its work with colleges, universities 
and programs, its processes and, above all, its values and the values of 
higher education. Given accreditation’s influence as higher education’s 
primary means of assuring and improving academic quality, much is 
at stake. 

Until recently, the role, key features and values of accreditation 
have been relatively stable, dating back to the formation of the first 
accrediting organizations in the late 19 th century. Accreditation was 
left to accrediting organizations and the colleges, universities and 
programs that created and funded these bodies for purposes of self- 
review and improvement. Professionals judged professionals. As a 
mechanism for internal accountability, accreditation was a complex 
and nuanced process, formative in judgment, relying often on 
qualitative analysis and carefully deliberative before pronouncing that 
institutional or programmatic efforts to improve were unsuccessful. 

As the number of accrediting organizations grew over time, 
the enterprise went from the review of a small group of public and 
private nonprofit colleges and universities to include more and 
more institutions and programs and the for-profit higher education 
sector. Eighty-five recognized accrediting organizations are currently 
operating, with more than 7,500 institutions and 20,000 programs 
enjoying accredited status. Institutional mission became central to 
judging quality and accreditation demonstrated significant flexibility 
and capacity for innovation in higher education, responding to the 
emergence of community colleges, the advent of distance learning, 
the powerful impact of technology on teaching and learning and, most 
recently, internationalization. 


But it has been the relationship into which accreditation entered 
with the federal government in the early 1950s that has proved most 
consequential with regard to accreditation’s stability and operation. 
Often referred to as “gatekeeping,” this relationship involves a 
partnership where accreditation is considered in federal law to be a 
“reliable authority” on academic quality, required for access to federal 
student aid and other funds. In order for a college, university or 
program to be eligible for federal money, it must be accredited by an 
institutional or programmatic accreditor that has been “recognized” 
by the federal government. Recognition is a process by which 
government, through the U.S. Department of Education (USDE), 
scrutinizes and judges accrediting organizations on a periodic basis 
for their capacity to serve as these reliable authorities. 


accreditation 

The current driving force for government with regard to 
accreditation is an expectation of robust public accountability. “Public 
accountability” is about higher education and accreditation answering 
to external actors such as government, employers, students and the 
public. It may be contrasted with the internal accountability that has 
been typical of higher education for centuries and is why accreditation 
was created. Public accountability means that internal accountability, 
academics reviewing academics as the hallmark of accreditation, 
is now perceived as insufficient. Higher education has always had 
an external, public accountability obligation. The change here is 
that external accountability has eclipsed internal accountability in 
importance. 



Public accountability and the reshaping of 
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Public accountability has emerged in an environment where 
“higher education” typically no longer evokes an image of a pristine 
college campus with young men and women studying full time. Higher 
education is now more often captured by a single parent who “goes to 
college” by sitting in front of the computer at 10:00 p.m. at the end 
of a long day of work. In this environment, the role of colleges and 
universities is to educate increasing numbers of the population for 
work and credentialing, is integrated into daily life and work, and is 
available full- and part-time 24/7 through online learning and other 
technologies. 

As a dominant theme of the federal government, public 
accountability is one of the few areas of agreement that cuts across 
political parties and is also embraced by the press and pundits. It 
includes expectations that higher education will enhance transparency, 
providing more readily accessible and easily understandable 
information to the public about the performance of institutions 
and achievement of students. It is about attention to what students 
accomplish as central to judgment about the worth of a college or 
university education. It is about the public and government being 
better informed about the use of public funds to finance higher 
education. 

The pressure on the accreditation enterprise is intense. The 
potential for significant transformation is great, in contrast to many 
years of a more placid pace of change that did not challenge the 
accreditation fundamentals - peer review, self-regulation, emphasis 
on institutional mission and responsible institutional independence 
as central to judging quality. 


This most recent public accountability emphasis began with 
the 2005-2006 Commission on the Future of Higher Education 
established by the U.S. Secretary of Education. The report issued 
by the commission was decidedly critical of both higher education 
and accreditation in a number of areas, including accountability. It 
contributed to the reshaping of accreditation by calls for greater 
transparency, especially with regard to student learning outcomes 
and information to the public. It offered a very public criticism of 
accreditation as weak and not improving quality. It urged much 
greater involvement of the government in academics. 

The commission report had a significant impact on the 
subsequent reauthorization of the Higher Education Act (2008) and 
the accompanying regulations issued in 2009, 2010, and 2011. The 
law’s revisions added to the reshaping of accreditation by expanding 
government authority over the conduct of the accreditation process: 
how appeals were handled, the extent of confidentiality and the 
opportunity for litigation. It moved the government into academic 
areas such as defining a student credit hour, requiring institutional 
transfer of credit policies and textbook policies and setting expectations 
of student achievement. At the same time, USDE was scrutinizing 
accrediting organization operation at a level of detail unseen in the 
past. Both the Congress and USDE were more heavily engaged in 
areas of academic judgments heretofore the traditional province of 
academic faculty. 

The net impact was a shifting of more and more responsibility and 
authority for academic quality from accreditors and institutions to the 
federal government. 
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What to do about the reshaping: Addressing 
misalignment 


The reshaping of accreditation has surfaced a significant 
misalignment between accreditation as it has been traditionally 
practiced and current public accountability demands. Unless this 
can be addressed, the core practices of accreditation as well as its 
fundamental values may be overtaken, perhaps replaced with a 
national system that may be standardized and relies primarily on 
quality review through government. 

It is not enough, apparently, for accreditation to (1) be answerable 
to institutions and programs that are reviewed (self-regulation), 
(2) make extensive use of peer review, (3) provide some but not all 
information to the public, (4) rely heavily on qualitative judgment 
about academic quality and (5) sustain a mission-based approach to 
quality that may not readily lend itself to standardization of expected 
results. Critics tend to overlook the value of accreditation especially 
with regard to quality improvement and to ignore the substantial 
contribution of accreditation to the growth and development of the 
higher education enterprise. 

In the name of public accountability, accreditation is expected to 
play a consumer protection role that is misaligned with a peer-driven, 
nuanced system of quality review focused on improvement of higher 
education. Accredited status signals the legitimacy and reliability 
of a college or university and builds public confidence about higher 
education. However, today’s expectations of consumer protection 


are about providing instant information, ideally electronically, that 
is summative, apparently uncomplicated, often quantitative and 
standardized, offering the consumer “bright lines” by which to make 
judgments. Rankings are an example. Accreditation, in contrast, 
relies on formative judgments. The standards call for evidence, often 
qualitative, that cannot be judged by bright lines and that relies on 
the mission of an institution and professional judgment, making 
standardization unrealistic. Consumer protection is important and 
valuable, but current expectations of up-or-down judgments are not 
effectively met using accreditation as the vehicle to achieve this goal. 

Doing something about reshaping accreditation means resolving 
this misalignment. Given the likely continued importance of public 
accountability, the challenge is to address means to respond to 
expectations - but in ways that do not diminish the valued core 
features of this enterprise. It is vital that the accreditation community 
take steps to at least strongly influence if not lead such changes. 
Actions in four areas are essential. 

First, we need to step up our advocacy for accreditation. 

Accreditation’s overall performance for more than a century has 
been part of the creation of a higher education enterprise that is 
unsurpassed in both access and quality, whatever its limitations. 
Peer review is effective. Self-regulation works. The values on 
which accreditation is built and which it reflects - the centrality of 
institutional mission, institutional autonomy and academic freedom - 
have been fundamental to the past successes of higher education and 
remain critical to future successes. If we do not make the case for the 
values and effectiveness of our enterprise, who will? 
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This advocacy needs to draw attention to the compelling role 
that accreditation plays in society. As the primary signal of the basic 
academic acceptability of colleges, universities or programs, students, 
the public, and the government rely heavily on accreditation. “Is it 
accredited?” is often the first question of prospective students and 
their families, employers, and foundations when deciding to attend or 
invest in a college or university. It is the first question of institutions 
examining the prior education of students seeking to transfer or to 
enter graduate school. As indicated above, accredited status is a 
requirement for receipt of federal funds, totaling some $175 billion 
per year at this time. 

The Council for Higher Education Accreditation (CHEA), for a 
number of years, has urged greater public awareness of the extent 
to which the society already relies on accreditation as one means to 
emphasize its value and credibility. The CHEA Initiative, a national 
dialogue on the future of accreditation underway since 2008, has 
worked to make the case for the richness and worth of accreditation 
review, emphasizing the depth and comprehensiveness of the 
process. At the same time, the Initiative discussions have stressed 
the difference between accreditation and the consumer protection 
expectations described above, committed to the view that forcing 
current expectations on accreditation will undermine its values 
and its effectiveness. Throughout the dialogue, CHEA has pressed 
accreditation to further address public accountability, but not at the 
press of diminishing reliance on peer review and the goal of quality 
improvement. 

Second, both the academic and accreditation 
communities need to emphasize the importance of our 


tradition of leadership for academic quality carried out at 
the institutional level. Public accountability, focusing on national 
or federal issues, is coming in conflict with the successful tradition 
of institutional leadership. Heretofore, we have turned to colleges 
and universities for what counts as effective education, judgment 
about student achievement and determination of paths to success in 
teaching, learning and research. With government oversight expanding 
into the academic arena, the role of institutions is diminished as 
government oversight replaces these judgments. The focus on public 
accountability at a federal or national policy level intensifies, thus 
ignoring institutions. Students, however, are not looking for national 
policy studies or aggregate national data about higher education. 
Instead, students want to know about specific institutions: “What will 
happen if I attend Fantastic U? Galactic U? What is the likelihood of 
graduating, transferring, getting a job?” 

Third, accreditation needs to more fully engage the public 
accountability imperative in order to influence it. We need a 
stronger voice. Efforts over the years to ignore the growing emphasis on 
public accountability - and there have been many - have proved futile. 
Initiatives that attempted to essentially finesse accountability using 
assessment have not succeeded. The public is interested in disclosure, 
not assessment. Disagreements within our own community have also 
been a significant barrier. A utilitarian approach to higher education 
now dominates and needs to be challenged. We are leaving our future 
up to others by refusing to lead. The price of failing to act with regard 
to accountability continues to grow. 

Society expects universal access to higher education. This 
expectation is accompanied by consumer-like interest in what higher 
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education is supposed to provide. Degrees are valued mainly to the 
extent to which they result in good-paying jobs, with less concern 
for expanding intellectual capacity. If colleges and universities are 
failing to graduate students or assist with the achievement of other 
education goals, they are viewed as inadequate. Where at one time the 
society valued higher education as an opportunity and valued open 
access even if it did not always result in academic success, this is no 
more. Two especially vital commitments are losing salience: the value 
of education as intellectual development and the value of educational 
opportunity for all, even when some will not succeed. 

As we take a major step toward greater responsiveness to public 
accountability, we need to find a way to maintain the benefits of 
accreditation, including our commitment to peer review and a mission- 
driven system, institutional autonomy and academic freedom. At the 
same time, we need to provide more information to the public about 
our performance and what counts as basic academic effectiveness. We 
need this framed in a manner that turns the tide with regard to the 
utilitarian approach to higher education that dominates the public 
accountability discussion. 

We can make significant progress with both emphasizing 
institutional leadership and engaging public accountability by, 
for example, each college or university agreeing to a short list of 
performance indicators, developing evidence with regard to success 
in achieving these results and publishing this information. This 
might include information about graduation, achievement of other 
educational goals, transfer, entry to graduate school and, where 
appropriate, job placement. This effort would assure more attention 


to student learning and transparency, two key demands of public 
accountability. It would provide the public with a better sense of the 
likely results of investing in a college education. It would not interfere 


judgment about student achievement where it belongs - with our 
faculty. 

Fourth, we need to reflect on the strategies we use 
to influence government policy. For the most part, both the 
academic and accreditation communities have approached the 
emphasis on public accountability by learning what government 
wishes to do and attempting to accommodate in a way that minimizes 
likely negative impact on what we do. This has not been working with 
regard to accreditation. The reshaping is too great, too fundamental 
for a more incremental approach. What is discussed above - a 
program of strong advocacy for accreditation, a renewed commitment 
to the role of institutional leadership, along with developing public 
accountability that works for accreditation - can figure effectively into 
our influencing. 


The academic and accreditation communities are confronted with 
the most substantial challenge to accreditation, perhaps in its history. 
The sixty years of accreditation’s relationship with government has 
reached a point where the government’s role is expanding significantly 
and reshaping the accreditation enterprise. During the past six years 
of concern with public accountability, the federal role in both judging 


with the work of faculty and their academic freedom. It would leave 
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academic quality and direct management of accreditation has grown 
to the point that it dominates traditional peer review and institutional 
leadership for academic quality. 

Public accountability carries expectations that accreditation 
will be first and foremost accountable to external actors rather than 
higher education community - professionals judging professionals. 
Discussions about accountability convey a strong preference for 
quantitative, summative, utilitarian-driven evaluation of quality 
in contrast to the typical formative, qualitative peer review that is 
characteristic of accreditation. It conveys a strong interest in national 
standards, if not standardization itself. This is incompatible with the 
fundamentals of accreditation. 

The accreditation and academic communities cannot allow this 
situation to develop further. Four actions can be helpful in seizing the 
initiative with regard to public accountability in order to assure that 
we do not pay the price of loss of the fundamental features and values 
of accreditation - institutional autonomy, academic freedom and 
institutional mission. We need advocacy for accreditation that reflects 
the urgency of the moment. We need to blunt the challenges to the 
robust role of institutional leadership for academic quality. We need to 
play a more effective role in framing what public accountability means 
and how it is carried out. Finally, we need to reflect on the strategies that 
we use to engage government in the public accountability discussion, 
seeking fresh approaches that enable to sustain accreditation while 
also responding to the environment in which higher education and 
accreditation are operating. 


Disciplining Peer Review: 
Addressing Some Deficiencies in U.S. 
Accreditation Practices 


— Peter T. Ewell 


The U.S. approach to assuring quality in higher education, 
embodied in the institutional accreditation process, differs 
substantially from the way this function is carried out in most other 
countries. For the most part, quality assurance elsewhere takes place 
under ministerial control - largely because most postsecondary 
institutions are public. Accreditation in the U.S. , in contrast, is at least 
nominally a voluntary process and is neither operated nor paid for 
by government. Similarly, the principal purpose of quality assurance 
elsewhere is to provide evidence that public money is spent effectively 
and to provide consumer information on the basis of which potential 
students can determine where to enroll. The principal purpose of U.S. 
accreditation is less clear because accreditors claim to simultaneously 
assure quality and to help institutions improve. Another significant 
difference between U.S. accreditation and quality assurance elsewhere 
is the central place of peer review. While peer review is used to some 
extent elsewhere, much of the substance of quality review outside the 
U.S. is undertaken by quality agency staff or by semi-professional 
reviewers. 
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I chose the topic of peer review for my contribution to Frank 
Murray’s Festschrift because TEAC’s use of an audit approach to 
quality review first got me thinking concretely about alternatives to 
U.S. accreditation’s well-worn design. I was dissatisfied with many 
aspects of peer review as the main basis for making judgments about 
academic quality and, through both direct experience and the work 
of such quality assurance scholars as David Dill and Bill Massy, was 
growing ever more familiar with the academic audit (Dill, Williams, 
Massy, and Cook, 1996). But I was equally convinced that peer 
review, done rigorously and well, both symbolized and embodied the 
assumption of collective professional responsibility for quality on 
the part of the academy and was a sound review technique in some 
important areas. My intent in this essay, therefore, is to briefly but 
systematically explore the merits and drawbacks of peer review as a 
quality assurance tool in the current higher education context and to 
suggest areas where the process can be improved. 1 


Although institutional accreditation has been around for more 
than a century in the U.S., the current dominant model of a mission- 
based process featuring a self-study followed by a site visit by a 


' My primary referent in this essay is institutional, not specialized/programmatic 
accreditation, primarily centered in the seven regional commissions. Specialized 
accreditors certainly make heavy use of peer review and might benefit from some 
of these observations and suggestions, but they are also beholden to specific 
stakeholder communities and have specific (often quantitative) standards of 
performance that buffer the absolute dominance of peers that now characterizes 
regional accreditation. 



Some background 


team of peers over a ten-year cycle evolved gradually from the 1930s 
through the 1950s (Ewell, 2008). This model, pioneered by the North 
Central Association, replaced review against quantitative criteria with 
minimum standards of performance that addressed degree awards, the 
breadth and depth of programs, faculty numbers and qualifications, 
library and academic support, physical facilities, and fiscal condition. 
Minimum standards such as these, of course, were relatively easy to 
apply and institutions frequently were examined by only one reviewer 
in the course of a single day. The shift to a new model centered on peer 
review was driven by a number of factors. First, the scope of regional 
accreditation had by this point expanded to include a far more diverse 
array of institutions including less selective colleges, normal schools, 
and (later) community colleges. This meant that a single set of 
quantitative standards were not flexible enough to support meaningful 
judgments of quality. At the same time, the posture of accreditation 
shifted gradually from a compliance-oriented minimum standards 
approach to one emphasizing institutional improvement. Peer 
review was, in theory, ideally situated to meet these new conditions 
because institutional diversity required reviewers drawn from all 
kinds of backgrounds and capable of bringing considerable practical 
experience to the process. At the same time, peer reviewers could use 
their experience to help institutions improve. 2 

The result was what some observers have called the “Golden 
Age” of accreditation during which regional accreditors operated 
an increasingly regularized process without any connection with 
government (Bloland, 2001). This situation shifted decisively with 


2 A change signaled by NCA’s beginning to call its peer reviewers “consultant 
evaluators” at that time. 
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passage of the Higher Education Act (HEA) of 1965 which, in essence, 
“deputized” accreditation to govern institutional eligibility for federal 
financial aid funds. Accordingly, accreditors have since then been 
mandated by the Department of Education to inspect institutional 
compliance with a steadily growing list of federal requirements. Peer 
review did not change. But the kinds of questions that peers were 
expected to pose about institutional condition and performance 
became steadily more complex, putting increasing strain on the 
process as a result. 


The case for peer review 

The observation has been made more than once that if the U.S. 
were establishing a national quality assurance system for higher 
education from scratch, we would not design the one based on 
regional accreditation that we now have. Would such a purpose-built 
alternative have a central role for peer review? In addressing this 
question, it is helpful to begin by reviewing the things that peer review 
does well and that ought to be preserved. 

The strongest case for review by peers is that they can bring to 
bear considerable expertise, drawn from experience, about what 
a “high quality” institution of higher education ought to look like. 
When the array of institutions under review was fairly homogeneous 
- as, arguably, it was until the 1980s - this was a powerful argument. 
Distance and asynchronous instructional delivery were in their infancy 
and proprietary institutions enrolled comparative small numbers of 
students. When atypical institutions were encountered, moreover, 


peer reviewers could be selected from institutions that resembled 
those under review. A related argument is that peer review provides 
a visible embodiment of the assumption of collective responsibility 
for self-governance owed by any profession that serves society. While 
this is far more active and visible in the realm of scholarship, where 
peer review is essential, it constitutes a persuasive case that the 
accreditation process assures the quality and integrity of the academy 
as a whole, beyond just guaranteeing the integrity of individual 
colleges and universities. 

Other arguments in support of peer review center on the quality 
improvement role of accreditation. If selected appropriately and 
allowed the opportunity to provide advice with candor, peer reviewers 
can actively help colleges and universities get better through 
constructive feedback. At the same time, peer reviewers can help 
disseminate good practices by taking what they learn in the course of 
a review back to their own campuses for local application. Although 
this does not always occur in the course of an accreditation visit, such 
efforts to cross-fertilize good practice are welcome and productive. 

Finally, an approach based on peer review is fairly cheap, at least 
with respect to direct cost. The bulk of the time invested by peer 
reviewers is contributed service as the regionals pay reviewers only 
nominal sums to undertake substantial commitments examining 
materials and visiting campuses. Alternative quality assurance systems 
based on professional reviewers, as are typical in other countries, 
must invest heavily in personnel costs and the costs associated with 
the development of a review infrastructures. 


93 


^ Drawbacks of peer review 

Though arguably well suited to an age when U.S. higher education 
was smaller and more homogeneous, at least two changes in the 
environment within which it must operate pose escalating challenges 
to accreditation’s heavy dependence on peer review. The first is 
technical: judging the quality of colleges and universities appropriately 
today requires levels of knowledge about important topics that 
typical peer reviewers do not possess. The second is political: in the 
age of heightened accountability best symbolized by the Secretary’s 
Commission on the Future of Higher Education (popularly known as 
the “Spellings Commission”), a process based on peer review looks like 
an inherent conflict of interest because those who judge performance 
are drawn from the same community that is being judged (U.S. 
Department of Education, 2006). Together, these challenges have 
combined to yield a number of specific drawbacks of peer review that 
are becoming ever more apparent. 

Accrediting organizations do take pains to match the characteristics 
of peer reviewers with those of the institutions that they will be asked 
to examine. But colleges and universities have become sufficiently 
complex organizations that it is difficult to find random members of 
the academic community who really understand how they function. 
Admittedly, the practicing administrators who generally constitute 
up to half of a given accreditation team have amassed a good deal of 
on-the-job knowledge about institutional functioning at their own 
institutions. But this background is not equivalent in either breadth 
or depth to that of an individual whose scholarship is focused on 
organizational or management effectiveness in college and university 


settings - for example, a faculty member in a Ph.D. program in Higher 
Education. Lack of technical background and expertise possessed by 
peer reviewers is even more acute in specific areas of institutional 
functioning like interpreting and acting on disaggregated graduation 
and retention data or evidence about the achievement of student 
learning outcomes. As these topics become ever more prominent 
in accreditation, accreditors are struggling to find peer reviewers in 
their regions with the technical background to examine them. Other 
specialized skill areas like finance or student recruitment, in contrast, 
have a much larger pool of institutional staff from which to draw. In 
parallel, most peer reviewers are not well suited to examine the many 
areas of compliance with federal regulations that the Department of 
Education currently expects of them. 

Part of the reason for this condition is that peer reviewers in 
U.S. accreditation receive relatively little dedicated training on how 
to conduct a review compared to the reviewers and auditors who 
staff quality assurance processes in other countries. Although this is 
beginning to change, most visiting team members only attend a day- 
long (or even half a day) orientation session before being deployed 
for review. This contrasts with the multi-day (and occasionally as 
long as a week) training regimens experienced by quality auditors in 
Europe or Australasia. As a result, peer review “training” in the U.S. 
context must necessarily be largely confined to familiarization with 
the accreditor’s standards and policies, and going over the specifics of 
the upcoming visit. 

Just as important as gaps in substantive knowledge, most peer 
reviewers lack many of the skills needed to conduct an effective 
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review. This is first apparent in most accreditation visits by the 
relatively rudimentary techniques that are typically used to gather 
evidence in the course of a site visit - direct inspection and document 
review, and group interviews conducted by dyads or triads of peer 
reviewers with institutional faculty and staff grouped by function. 
While both may be guided by general questions prepared in advance 
by the team, they are far less sophisticated than the kinds of research 
protocols used in sociological or anthropological studies. An equally 
important set of skills centers on interpreting evidence and coming 
to conclusions about the extent to which quality standards are met. 
One problem here is that most peer review teams are overscheduled 
when they visit campuses and, as a result, fail to devote the necessary 
time to discussions amongst themselves about the evidence they have 
collected and what it means. Another problem is the fact that when 
these discussions do occur, they tend to be unscripted. Consequently, 
vocal team members with strong opinions tend to get more air time 
than their equally informed, but more reticent, colleagues and strong 
team chairs can exert undue influence on conclusions simply by virtue 
of their prominence. 

Uneven knowledge and limited training contribute to another 
significant deficiency of the use of peer review in accreditation: lack 
of consistency in the outcomes of reviews. The perception that the 
outcomes of accreditation visits depend more on the composition of 
the review team than the actual elements of institutional condition 
or performance being examined is widespread among institutional 
leaders, for example (CHEA, 2006). Especially prevalent is the 
complaint that team members hold unfavorable views of an 
institution’s arrangements or practices if they differ markedly from the 


way these things look “back home” at the reviewers’ own institutions. 
These observations underscore the vulnerability of any assessment 
process based on individual perception conducted in the absence of 
explicit decision criteria or rubrics: the magnitude of variation across 
observers frequently renders the process unacceptably unreliable. 

A final deficiency of the use of peer review in accreditation is that the 
evidence that fellow academics provide is not balanced by viewpoints 
outside the academy. In contrast, external viewpoints are common 
in the quality assurance regimens applied to venues like health care 
or financial services. One important group that is missing in higher 
education review comprises stakeholders. These include students, 
employers who will hire the institution’s graduates, and the graduate 
schools who will admit them for further study. A similarly important 
group is members of the public, who count on the competence of a 
given college’s graduates in a range of roles and settings. To be sure, 
all institutional accrediting commissions have a few public members. 
But their role is too limited to overshadow widespread and long 
standing concerns that peer review is “inside baseball” and is riddled 
with conflict of interest (Newman, 1973). 


^ What could be done? 

Some observers have claimed that peer review’s significant 
deficiencies render it unsuitable as a quality assurance tool. But I 
believe that dropping it would go too far because, done well, it has 
much to contribute to both the practice of accreditation and the public 
perceptions of professional self-regulation on which widespread trust 
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in the academy depend. Instead, calling on institutions and accrediting 
organizations to take specific steps to improve the peer review process 
- to “discipline” it, if you will — might help alleviate its most prominent 
deficiencies. The most important of these steps include: 

• Training. Not surprisingly, I believe that more extensive 
training of peer reviewers would help a lot. Bringing the 
amount and frequency of training events up to the level 
typically experienced by auditors employed by the Quality 
Assurance Agency (QAA) in the United Kingdom, for example 
- multi-day events including simulations and role-playing 
exercises - would be a good first step. And training of this 
kind should not be a one-time event, but should instead 
involve periodic “refresher” sessions conducted in depth 
every few years, as well as one-day reorientation sessions that 
should occur before teams are deployed on a visit. In addition 
to extending the duration of training events, moreover, the 
focus of training needs to be more active and intentional. 
Instead of just covering topical content like what an 
accreditor’s standards and policies are about, training events 
need to devote considerable time to actively pract icing 
common review techniques. These should include directed 
interviewing and participant observation, as well as examining 
typical institutional documents like strategic plans, committee 
minutes, or assessment reports. Combinations of face-to- 
face and on-line training events could allow participation 
in training to be enhanced and expanded without excessive 
additional cost. 


• Reorganize review processes. Peer reviewers are also 
better at some things than others. As a result, it makes a 
lot of sense to restructure review processes to deploy peer 
reviewers on topics like faculty capacity and curriculum 
organization where faculty peer reviewers can be assumed 
to be knowledgeable, and use expert reviewers instead 

to examine topics like student success (retention and 
graduation) or the assessment of student learning outcomes 
where the typical faculty peer reviewer has less knowledge 
and background. This is already done in the area of financial 
adequacy, where dedicated members of the review team with 
special backgrounds in public or private institutional finance 
are typically used. Following this advice, several regional 
accreditors are establishing expert review panels on some 
of these topics . 3 These panels conduct parallel reviews of 
selected topics off-site and report their judgments to the 
commission directly. 

• Clearly distinguish quality assurance from 
institutional improvement. Institutional accreditation 
in the U.S. has always claimed a dual role for the process: 
it examines and certifies quality, while the review process 
provides important feedback on the basis of which institutions 
can improve what they do. If chosen appropriately to fit the 
topics or domains addressed by a self-study, peer reviewers 
are a good deal more appropriate and helpful in the role 


3 The most prominent current examples are the Southern Association of Colleges and 
Schools (SACS) and the Senior Commission of the Western Association of Schools 
and Colleges (WASC). 
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of providing feedback than they are in assuring quality. 
Accordingly, clearly distinguishing which aspects of a 
given review are assigned to which role would help those in 
charge of a review assign tasks. In my view, consulting and 
providing feedback can appropriately be assigned to peer 
reviewers, while more bright-line quality judgments are more 
appropriately guided by statistical performance measures 
conducted by expert review panels. 

Increase staff presence in on-site reviews. Agency 
staff members accompany peer review teams when they visit 
institutions at all accrediting organizations but the frequency 
with which this occurs varies a good deal. Some accreditors 
include staff on almost all site visits, while others do so for 
only a few visits or for only the first day of a multi-day site 
visit. Including staff is generally a good idea because their 
greater knowledge and experience with what happens in the 
course of a review can enable them to focus team attention on 
the right issues, can return a wandering team to the central 
task, or can help the team interpret evidence. Just as a “jury 
of peers” in a court of law requires advice and occasional 
intervention by a presiding judge, staff presence during a site 
visit can help guide the process of coming to judgment about 
the results of a review. 

Create more and better tools for reviewing quality. 

As noted earlier, most peer review teams use only a couple 
of relatively simple techniques like group interviews or 
document inspection to gather evidence during site visits. 

The effectiveness of these could be improved by developing 


protocols or rubrics and training observers to use them. In 
addition, the reach and reliability of evidence gathering could 
be greatly enhanced by adopting additional observational 
tools drawn from the social sciences. For example, “mini- 
surveys” consisting of a few closed-ended questions can be 
deployed in large group meetings at the outset, with results 
tabulated quickly by one member of the team and posted 
to better structure the discussion. This helps to avoid the 
dominance of only a few voices that tends to occur during 
such sessions. Similarly, fieldwork protocols based on 
anthropological research can be developed for use by a 
designated member of the team, whose assignment is to 
circulate on the campus systematically to observe typical 
interactions and practices. A parallel set of tools could be 
devised on how to organize site visits. One topic here might be 
guidance on how the team should budget its time on campus 
to ensure that there are sufficient opportunities for reflection, 
discussion, and coming to judgment. Another topic might be 
procedures for drawing conclusions about various aspects of 
institutional condition or performance based on approaches 
like nominal group technique that allow the team to move 
beyond “groupthink” and opinion dominance by a few vocal 
members. 

A non-trivial objection that can be made to adopting any of these 
enhancements to the review process is that they would add cost. But 
membership costs and costs associated with reviews already vary 
substantially across regional accreditors. This suggests that some 
institutions appear to be willing to bear higher costs so long as they 
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believe they are receiving accurate and reliable reviews conducted 
using more elaborate evidence-gathering procedures. Moreover, 
many of these costs could be covered by reallocating expenses away 
from components of the current institutional accreditation process 
that may add little value. For example, the traditional voluminous 
self-study consumes significant amounts of staff time for institutions 
under review, but is largely descriptive; in many regions it is already 
being displaced by much more focused reports on various topics 
organized around concrete evidence. Consequent savings in staff time 
could allow institutions to afford modest increases in membership 
fees and visit costs. 


Most aspects of higher education are notoriously difficult to change 
and accreditation is no exception. So while some experimentation 
is beginning to occur in response to widespread complaints about 
accreditation’s cost and its perceived shortfalls in discharging an 
increasingly prominent role in accountability, some thought needs to 
be devoted to how the recommended changes in peer review outlined 
above can best be implemented. 

First, despite the surface appeal of externally-induced “blunt 
instrument” approaches advanced through the recognition process 
administered by the National Advisory Committee on Institutional 
Quality and Integrity (NACIQI) or the negotiated rule-making process 
associated with the HEOA, I believe that it is best to let the academy 



Getting there 


take the initiative. Most college and university leaders now believe 
that proactive change is in the best interest of their institutions and 
of the academy as a whole, lest unwanted and ill-advised changes be 
mandated from the outside. The Task Force on Accreditation convened 
by the American Council on Education (ACE), for example, calls upon 
the national presidential associations to ask the regional accreditation 
organizations to implement these changes, even though the result 
may be a more rigorous review process that may uncover institutional 
shortcomings (ACE, 2012). At least half the regionals, meanwhile, 
have rolled out new standards and review processes. Some of these 
involve substantial changes in the role of peer review and more are 
under consideration. 

Second, consistent with one of the recommendations of the ACE 
Task Force, much could be accomplished by accreditors examining one 
another’s practices and engaging in mutually beneficial dialogue about 
how to make them better (ACE, 2012). Although I believe strongly that 
review criteria and practices need to be better aligned across regions, 
one virtue of having seven independent commissions is that it allows 
a lot of experimentation to take place. For example, at least one of 
the regionals - the Senior Commission of the Western Association of 
Schools and Colleges (WASC) has already adopted a review approach 
that assigns the task of examining institutional performance with 
respect to student success (retention and graduation rates) and 
student learning outcomes to expert panels in preference to peer 
reviewers. If this practice proves successful and if other accreditors 
can learn about it, other regionals might well change their practices 
in this direction. In the past, dialogue and mutual adoption has led to 
greater commonality. For example, all seven regionals have adopted 
review criteria originally developed by the Western Cooperative for 
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Educational Technology (WCET) and they have all aligned their 
criteria for institutional eligibility. 

Finally, establishing a mechanism for accreditors to document 
and disseminate good practices on a national basis would help a lot. 
Given its mission, the Council for Higher Education Accreditation 
(CHEA) would be a good host for this and CHEA has already played 
this role in a couple of other arenas. One such project proposed 
a set of documented data elements, together with definitions and 
recommended sources, which all accreditors could use for institutional 
reporting (CHEA, 2000). Extending this by creating an easy-to-access 
“toolkit” containing techniques intended to enhance the effectiveness 
of peer review, together with orientation and training sessions given 
at CHEA and regional annual meetings, might constitute a helpful 
contribution. 

Neither of these two suggestions, of course, guarantees widespread 
adoption of these practices and pursuing them remains a set of decisions 
that must be made by each accrediting organization independently. 
But better documentation and enhanced and dissemination of these 
practices on a more systematic and coordinated national basis appears 
promising. 


Final thoughts 

Peer review as an element of higher education quality assurance 
and as a central feature of institutional accreditation in the U.S. has 
vociferous supporters and critics. The former see peer review as the 
symbolic embodiment of self-regulation and as an effective tool for 


assuring quality as well as guiding improvement. The latter maintain 
that peer review is increasingly out of date given vast increases in the 
complexity of colleges and universities and would like to see it de- 
emphasized in favor of more “professional” approaches. My own view 
is in between. Peer review is good at some things and not very good 
at others. By carefully choosing the aspects of institutional condition 
and performance at which it is directed to match its strengths, and 
by designing tools to bolster its effectiveness in settings where this 
is possible, this time honored centerpiece of U.S. accreditation can 
indeed be “disciplined.” 
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